-- Logs begin at Tue 2022-06-21 17:13:20 UTC, end at Tue 2022-06-21 18:52:53 UTC. -- Jun 21 18:04:20.122492 dv-cinder-831247-19-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 21 18:04:20.131743 dv-cinder-831247-19-iscsi devstack@keystone.service[80602]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 21 18:04:20.133966 dv-cinder-831247-19-iscsi devstack@keystone.service[80602]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Jun 21 18:04:20 2022] *** Jun 21 18:04:20.133966 dv-cinder-831247-19-iscsi devstack@keystone.service[80602]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 21 18:04:20.133966 dv-cinder-831247-19-iscsi devstack@keystone.service[80602]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 21 18:04:20.133966 dv-cinder-831247-19-iscsi devstack@keystone.service[80602]: nodename: dv-cinder-831247-19-iscsi Jun 21 18:04:20.133966 dv-cinder-831247-19-iscsi devstack@keystone.service[80602]: machine: x86_64 Jun 21 18:04:20.133966 dv-cinder-831247-19-iscsi devstack@keystone.service[80602]: clock source: unix Jun 21 18:04:20.133966 dv-cinder-831247-19-iscsi devstack@keystone.service[80602]: pcre jit disabled Jun 21 18:04:20.133966 dv-cinder-831247-19-iscsi devstack@keystone.service[80602]: detected number of CPU cores: 4 Jun 21 18:04:20.133966 dv-cinder-831247-19-iscsi devstack@keystone.service[80602]: current working directory: / Jun 21 18:04:20.133966 dv-cinder-831247-19-iscsi devstack@keystone.service[80602]: detected binary path: /usr/bin/uwsgi-core Jun 21 18:04:20.133966 dv-cinder-831247-19-iscsi devstack@keystone.service[80602]: your processes number limit is 47680 Jun 21 18:04:20.133966 dv-cinder-831247-19-iscsi devstack@keystone.service[80602]: your memory page size is 4096 bytes Jun 21 18:04:20.133966 dv-cinder-831247-19-iscsi devstack@keystone.service[80602]: detected max file descriptor number: 2048 Jun 21 18:04:20.133966 dv-cinder-831247-19-iscsi devstack@keystone.service[80602]: lock engine: pthread robust mutexes Jun 21 18:04:20.135406 dv-cinder-831247-19-iscsi devstack@keystone.service[80602]: thunder lock: enabled Jun 21 18:04:20.135406 dv-cinder-831247-19-iscsi devstack@keystone.service[80602]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 21 18:04:20.135406 dv-cinder-831247-19-iscsi devstack@keystone.service[80602]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 21 18:04:20.177853 dv-cinder-831247-19-iscsi devstack@keystone.service[80602]: Python main interpreter initialized at 0x5637f78619c0 Jun 21 18:04:20.177853 dv-cinder-831247-19-iscsi devstack@keystone.service[80602]: python threads support enabled Jun 21 18:04:20.177853 dv-cinder-831247-19-iscsi devstack@keystone.service[80602]: your server socket listen backlog is limited to 100 connections Jun 21 18:04:20.177853 dv-cinder-831247-19-iscsi devstack@keystone.service[80602]: your mercy for graceful operations on workers is 90 seconds Jun 21 18:04:20.178260 dv-cinder-831247-19-iscsi devstack@keystone.service[80602]: mapped 403077 bytes (393 KB) for 2 cores Jun 21 18:04:20.178348 dv-cinder-831247-19-iscsi devstack@keystone.service[80602]: *** Operational MODE: preforking *** Jun 21 18:04:20.178558 dv-cinder-831247-19-iscsi devstack@keystone.service[80602]: *** uWSGI is running in multiple interpreter mode *** Jun 21 18:04:20.178558 dv-cinder-831247-19-iscsi devstack@keystone.service[80602]: spawned uWSGI master process (pid: 80602) Jun 21 18:04:20.178768 dv-cinder-831247-19-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 21 18:04:20.179720 dv-cinder-831247-19-iscsi devstack@keystone.service[80602]: spawned uWSGI worker 1 (pid: 80614, cores: 1) Jun 21 18:04:20.179823 dv-cinder-831247-19-iscsi devstack@keystone.service[80602]: spawned uWSGI worker 2 (pid: 80615, cores: 1) Jun 21 18:04:20.180974 dv-cinder-831247-19-iscsi devstack@keystone.service[80602]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 21 18:04:25.821412 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 21 18:04:25.821412 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: from cryptography.utils import int_from_bytes Jun 21 18:04:25.822644 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 21 18:04:25.822644 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: from cryptography.utils import int_from_bytes Jun 21 18:04:25.822852 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 21 18:04:25.822852 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: from cryptography.utils import int_from_bytes Jun 21 18:04:25.822852 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 21 18:04:25.822852 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: from cryptography.utils import int_from_bytes Jun 21 18:04:25.875321 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 21 18:04:25.875599 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 21 18:04:25.875756 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] command line args: [] {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 21 18:04:25.875911 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 21 18:04:25.876045 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 21 18:04:25.876174 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] ================================================================================ {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 21 18:04:25.876294 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 21 18:04:25.876487 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] admin_token = **** {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.876621 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] command line args: [] {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 21 18:04:25.876761 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] config_dir = [] {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.876888 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.877011 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 21 18:04:25.877141 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] config_source = [] {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.877259 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] ================================================================================ {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 21 18:04:25.877383 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] debug = True {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.877383 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.877747 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] admin_token = **** {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.877881 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.878003 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.878118 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] config_dir = [] {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.878236 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.878406 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.878523 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.878640 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] list_limit = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.878749 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] config_source = [] {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.878866 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] log_config_append = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.878966 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.879078 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] debug = True {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.879187 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] log_dir = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.879311 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] log_file = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.879428 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.879574 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.879789 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.879920 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.880037 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.880135 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.880285 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.880500 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.880729 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.880986 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.880986 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.881416 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.881717 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.881717 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.882213 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] list_limit = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.882495 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.882744 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] log_config_append = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.883024 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.883307 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.883604 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.883604 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.884083 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] log_dir = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.884356 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.884745 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] log_file = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.884989 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.885274 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.885274 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.886269 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.23/identity {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.886269 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] publish_errors = False {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.886587 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.886823 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.886823 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.887171 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.887405 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.887584 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.887776 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.887776 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.888131 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.888315 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] standard_threads = False {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.888522 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.888743 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.888743 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.889101 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.889359 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.889359 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] use_journal = False {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.889701 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.889886 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] use_json = False {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.889886 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] use_stderr = False {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.889886 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] use_syslog = False {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.889886 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.890576 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.890789 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.890984 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.891186 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.891459 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.891650 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.891650 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.892001 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.893002 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.893002 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.893260 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.893402 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.893402 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.893630 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.893774 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.893774 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.893774 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.893774 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.894234 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.23/identity {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.894481 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.894616 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] publish_errors = False {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.894755 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.894755 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.894991 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.895124 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.895124 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.895363 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.895491 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.895624 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.895750 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.895750 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.896010 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.896153 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.896281 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.896434 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.896580 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] standard_threads = False {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.896729 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.896872 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.897010 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.897010 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.897258 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.897396 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.897539 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.897684 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.897820 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] use_journal = False {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.897952 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.897952 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.898182 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] use_json = False {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.898331 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.898448 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] use_stderr = False {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.898562 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.898681 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] use_syslog = False {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.898807 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.898934 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:04:25.899066 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.899066 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.899290 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.899425 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.899425 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.899678 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.899809 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.899931 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.900044 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.900044 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.900265 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.900384 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.900384 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.900777 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.900907 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.901038 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.901172 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.901172 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.901472 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.901594 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.901707 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.901819 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.901819 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.902038 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.902151 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.902151 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.902392 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.902488 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.902488 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.902680 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.902791 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.902907 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.903017 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.903017 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.903232 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.903352 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.903352 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.903540 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.903636 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.903738 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.903849 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.903849 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.904038 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.904133 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.904133 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.904304 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.904520 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.904520 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.904520 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.904841 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.904953 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.904953 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.905147 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.905319 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.905525 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.905718 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.905718 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.906074 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.906203 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.906203 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.906443 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.906557 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.906557 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.906755 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.906942 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.906942 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.907163 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.907353 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.907353 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.907641 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.907775 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.907976 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.908119 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.908119 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.908457 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.908588 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.908588 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.908868 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.909032 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.909032 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.909283 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.909401 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.909401 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.909726 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.909858 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.910031 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.910149 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.910149 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.910464 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.910586 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.910586 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.910866 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.910982 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.910982 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.911222 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.911346 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.911346 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.911571 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.911683 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.911795 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.911961 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.911961 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.912196 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.912331 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.912331 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.912688 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.912814 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.912938 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.913080 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.913080 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.913277 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.913389 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.913511 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.913628 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] auth.external = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.913628 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.913962 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.914079 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.914197 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.914323 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.914323 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] auth.password = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.914567 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.914694 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] auth.token = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.914694 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.914953 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.915080 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.915209 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.915336 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.915336 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.915581 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.915707 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.915843 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.916022 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.916022 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.916278 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.916452 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] credential.caching = True {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.916452 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.916661 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.916769 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.916882 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.917507 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.917507 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.917686 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.917784 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.917898 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.917997 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.917997 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.918190 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.918359 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.918359 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.918853 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.918973 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.918973 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.919194 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.919316 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.919429 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.919540 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.919652 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.919786 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.919911 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.920045 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.920045 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] federation.caching = True {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.920369 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.920583 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.920583 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.920835 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.920953 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.920953 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.921156 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.921271 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.921405 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.921535 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.921535 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.921774 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.921896 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.921896 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.922140 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.922269 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.922397 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.922525 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.922525 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] identity.caching = True {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.922767 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.922943 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.922943 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.923202 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.923332 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.923462 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.923586 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.923586 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.923820 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.923974 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.924089 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.924201 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.924319 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.924581 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.924707 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.924842 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.924973 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.925090 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.925204 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.925316 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.925431 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.925680 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.925800 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.925929 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.926124 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.926251 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.926418 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.926554 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.926698 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.926829 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.926968 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.927170 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.927296 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.927431 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.927594 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.927727 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.927855 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.927970 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.928098 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.928313 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.928544 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.928671 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.928790 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.928922 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.929068 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.929298 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.929421 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.929549 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.929686 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.929824 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.929956 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.930096 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.930225 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.930383 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.930517 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.930693 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.930827 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.930968 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.931159 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.931293 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.931423 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.931557 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.931688 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.931865 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.932005 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.932140 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.932308 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.932493 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.932708 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.933331 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.933470 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.933707 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.933841 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.933965 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.934089 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.934242 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.934374 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.934501 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.934630 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.934796 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.934952 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.935088 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] ldap.user = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.935206 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.935333 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.935515 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.935633 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.935745 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.935978 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.936101 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.936229 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.936353 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.936561 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.936732 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.936867 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.936997 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.937195 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.937330 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.937462 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.937633 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.937769 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.937900 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.938025 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] auth.external = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.938293 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.938415 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.938545 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.938661 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.938826 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.938945 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.939057 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.939250 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.939397 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.939525 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.939639 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] auth.password = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.939745 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.939998 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] auth.token = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.940136 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.940279 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.940500 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.940651 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.940781 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.940909 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.941034 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.941183 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.941315 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.941456 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.941587 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.941721 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.941931 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.942128 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.942315 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.942503 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.942683 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.942683 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.943038 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.943233 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.943417 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] credential.caching = True {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.943604 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.943804 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.943987 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.943987 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.944319 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.944514 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] resource.caching = True {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.944693 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.944877 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.945062 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.945247 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.945451 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.945622 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.945622 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.945951 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.946127 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.946311 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.946519 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.946700 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.946884 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.946884 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.947234 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.947413 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.947595 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.947762 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] role.caching = True {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.947762 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] role.driver = sql {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.948072 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.948824 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.949019 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.949229 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.949229 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.949572 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.949745 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.949925 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.950107 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.950305 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.950487 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.950670 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] federation.caching = True {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.950858 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.950858 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.951183 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.951349 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.951515 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.951681 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.951865 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.952035 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.952222 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.952425 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.952425 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.952774 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.952953 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.953139 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.953139 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.953491 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.953491 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.953826 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.954014 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.954189 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.954375 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.954552 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.954734 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.954923 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.955105 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.955105 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.955472 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] identity.caching = True {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.955655 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.955842 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.956022 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.956200 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.956483 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.956483 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.956850 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.957042 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.957231 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.957409 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.957586 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.957784 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.957970 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.958147 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.958147 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.958502 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.958688 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.958881 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.959063 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.959244 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.959431 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.959431 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] token.caching = True {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.959757 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.959950 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.960136 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.960320 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.960523 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.960702 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.960702 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.961070 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.961261 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.961449 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.961639 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.961831 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.962019 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.962019 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.962484 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.962669 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.962851 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.963022 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.963211 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.963419 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.963671 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.964612 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.964612 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.964941 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.965134 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.965315 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.965496 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.965682 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.965862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.966032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.966213 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.966393 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.966563 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] database.connection = **** {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.966757 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.966953 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.966953 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.967301 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.967492 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.967701 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.967887 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.968080 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.968257 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.968257 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.968615 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.968814 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.969001 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.969188 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.969371 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.969552 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.969552 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.969917 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.970108 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.970286 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.970476 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.970668 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.970854 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.970854 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.971224 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.971659 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.971849 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.972034 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.972221 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.972480 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.972659 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.972847 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.972847 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 21 18:04:25.973199 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.973199 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.973199 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.973199 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.973199 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.973199 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.973199 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.973199 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.973199 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.974760 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.974760 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.974760 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.974760 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] ldap.user = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.974760 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.974760 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.974760 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.974760 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.974760 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.976243 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.976243 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.976243 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.976243 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.976243 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.976243 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.976243 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.976243 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.976243 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.977778 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.977778 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.977778 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.977778 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.977778 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.977778 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.977778 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.977778 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.977778 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.979246 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.979246 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.979246 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.979246 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.979246 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.979246 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.979246 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.979246 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.979246 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.980801 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.980801 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.980801 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.980801 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.980801 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] resource.caching = True {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.980801 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.980801 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.980801 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.982099 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.982099 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.982099 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.982099 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.982788 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.982788 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.982788 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] role.caching = True {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.983293 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] role.driver = sql {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.983489 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.983710 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.984016 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.984319 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.984634 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.984938 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.985236 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.985533 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.985830 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.986108 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.986400 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.986693 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.986985 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.987274 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.987762 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.988037 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.988331 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.988687 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.988989 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.989291 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.989582 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.989890 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.990194 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.990480 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.990762 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.991042 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.991327 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.991628 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.991934 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.992233 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.992555 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.992842 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.993114 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.993399 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] token.caching = True {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.993689 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.993984 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.994264 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.994563 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.994857 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.995660 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.995950 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.996238 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.996580 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.996767 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:25.996963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:25.997143 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.997311 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:25.997502 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.997502 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.997819 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.998100 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.998380 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.998672 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.999017 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.999329 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] database.connection = **** {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.999623 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:25.999922 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:26.000210 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:26.000537 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:26.000825 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:26.001108 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:26.001389 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:26.001668 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:26.001965 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:26.002245 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:26.002534 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:26.002813 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:26.003299 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:26.003583 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:26.003872 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:26.004149 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:26.004472 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:26.004769 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:04:26.004989 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 21 18:04:26.006184 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:26.032736 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:26.033000 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:26.033330 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:26.035735 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:26.035967 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:26.043161 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:26.062638 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:26.062926 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:26.186209 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:26.199101 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:26.238586 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:26.238851 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:26.242585 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:26.242939 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:26.310148 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:26.310415 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:26.310580 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:26.310804 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:26.311024 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:26.311167 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:26.311310 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:26.311487 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:26.311860 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:26.312153 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:26.312463 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:26.312758 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:26.337954 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:26.338198 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:26.338398 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:26.338554 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:26.338683 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:26.339184 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:26.339530 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:26.339659 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:26.339797 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:26.339922 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:26.340066 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:26.340066 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:26.340066 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:26.340535 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:26.351301 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:26.351938 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:26.365302 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:26.365545 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:26.365754 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:26.365980 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:26.366169 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:26.366304 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:26.366428 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:26.366613 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:26.366759 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:26.366982 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:26.367279 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:26.367597 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:26.367885 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:26.368168 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:04:26.385838 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.386091 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 21 18:04:26.386273 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.386405 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.387224 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 21 18:04:26.387405 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 21 18:04:26.387405 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.387706 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.387838 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.387962 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 21 18:04:26.388093 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.388224 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.388352 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.388561 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.388561 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.388561 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.398804 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.399094 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.399367 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.399655 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.400644 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.401026 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.401521 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.402292 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.408847 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80614) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:04:26.410885 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80615) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:04:26.412220 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80614) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:04:26.412595 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.412881 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.413167 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.413452 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.413718 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.413990 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.414265 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.414659 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.414951 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80615) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:04:26.415095 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.415245 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.415380 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.415800 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.416151 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.416544 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.416874 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.418376 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.418376 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.418756 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.419071 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.419408 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.444149 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.445763 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80614) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:04:26.446044 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.447889 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.449710 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80615) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:04:26.450053 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.451491 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80614) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:04:26.451923 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.452210 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.455912 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80615) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:04:26.456275 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.456662 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.459876 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80614) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:04:26.460176 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.465047 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80615) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:04:26.465365 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80614) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:04:26.465523 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.465648 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.465648 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.465923 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.466148 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.466431 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.466711 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.466983 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.470506 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80615) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:04:26.470864 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.471209 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.471836 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.472176 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.472552 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.472888 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.473259 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.487354 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.487651 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.492527 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80614) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:04:26.493477 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.493828 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.497048 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.497480 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.499403 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80615) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:04:26.502803 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80614) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:04:26.503602 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.503993 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.506662 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80614) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:04:26.509870 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80615) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:04:26.511726 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.512437 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.512818 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.513158 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.513395 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.513682 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.515692 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80615) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:04:26.519718 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.520087 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.520481 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.520846 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.521190 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.521543 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.534412 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80614) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:04:26.534711 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.534980 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.535254 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.543656 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80615) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:04:26.544033 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.544251 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.544429 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.544789 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.546195 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.546559 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.550135 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80614) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:04:26.550444 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.550753 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.551064 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.551407 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.553781 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.555654 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.556000 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.560020 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80615) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:04:26.560379 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.560773 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.561126 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.561473 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.565786 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80614) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:04:26.566083 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.566375 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.566658 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.568461 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.568759 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.569064 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.576380 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80615) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:04:26.576793 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.577150 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.577503 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.577853 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.578202 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.578551 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.587700 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80614) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:04:26.591279 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80614) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:04:26.594826 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.596985 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.598725 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80614) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:04:26.599017 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.599302 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.600089 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80615) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:04:26.604025 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80615) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:04:26.607817 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.608426 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.609626 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80614) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:04:26.610371 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.612110 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80615) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:04:26.612503 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.612861 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.613015 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.613241 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.613658 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.613938 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.621541 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.623488 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80615) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:04:26.623978 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80614) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:04:26.624288 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.624612 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.627267 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.627647 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.628000 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.628349 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.632911 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80614) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:04:26.633211 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.633496 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.633776 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.634037 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.634435 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.634721 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.634988 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.635272 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.635551 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.636348 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.636638 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.636907 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.637280 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.639998 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80615) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:04:26.640437 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.641065 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.649481 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80615) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:04:26.649853 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.650200 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.650541 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.650883 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.651239 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.652365 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.652579 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.652929 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.653266 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.653608 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.653942 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.654433 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.654789 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:04:26.676448 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5637f78619c0 pid: 80614 (default app) Jun 21 18:04:26.680084 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f4d271-6f6c-4efd-9d55-22480af99436 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:26.680381 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f4d271-6f6c-4efd-9d55-22480af99436 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:26.680704 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f4d271-6f6c-4efd-9d55-22480af99436 None None] PATH_INFO: `/v3` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:26.685112 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1/1] 10.222.0.23 () {50 vars in 791 bytes} [Tue Jun 21 18:04:26 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:04:26.699324 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5637f78619c0 pid: 80615 (default app) Jun 21 18:04:35.612744 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb36d2-aea9-495b-a17f-88479addfe38 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:35.613346 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb36d2-aea9-495b-a17f-88479addfe38 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:35.613346 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb36d2-aea9-495b-a17f-88479addfe38 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:35.614633 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 2/2] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:04:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:04:35.622864 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7b400f-032d-40ce-86d2-464ffece70c7 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:35.623372 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7b400f-032d-40ce-86d2-464ffece70c7 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:35.623468 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7b400f-032d-40ce-86d2-464ffece70c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:35.694016 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: INFO dbcounter [None req-9a7b400f-032d-40ce-86d2-464ffece70c7 None None] Registered counter for database keystone Jun 21 18:04:35.717423 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG oslo_db.sqlalchemy.engines [None req-9a7b400f-032d-40ce-86d2-464ffece70c7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80615) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 21 18:04:35.724412 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writer thread running {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 21 18:04:35.906586 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG passlib.handlers.bcrypt [None req-9a7b400f-032d-40ce-86d2-464ffece70c7 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80615) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 21 18:04:35.907014 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG passlib.handlers.bcrypt [None req-9a7b400f-032d-40ce-86d2-464ffece70c7 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80615) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 21 18:04:35.962358 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-9a7b400f-032d-40ce-86d2-464ffece70c7 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:04:35.964671 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-9a7b400f-032d-40ce-86d2-464ffece70c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:04:36.091150 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1/3] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:04:35 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 472 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 21 18:04:36.217703 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fb36d2-aea9-495b-a17f-88479addfe38 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:36.290915 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: INFO dbcounter [None req-75fb36d2-aea9-495b-a17f-88479addfe38 None None] Registered counter for database keystone Jun 21 18:04:36.313112 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG oslo_db.sqlalchemy.engines [None req-75fb36d2-aea9-495b-a17f-88479addfe38 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80614) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 21 18:04:36.319575 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writer thread running {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 21 18:04:36.442616 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fb36d2-aea9-495b-a17f-88479addfe38 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:36.446889 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cc7a38-170f-486f-83d5-7bc427700f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:36.447547 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc7a38-170f-486f-83d5-7bc427700f93 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:36.447830 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc7a38-170f-486f-83d5-7bc427700f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:36.448741 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc7a38-170f-486f-83d5-7bc427700f93 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:36.457453 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cc7a38-170f-486f-83d5-7bc427700f93 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:04:36.510229 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: WARNING keystone.common.rbac_enforcer.enforcer [None req-45cc7a38-170f-486f-83d5-7bc427700f93 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 21 18:04:36.510513 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cc7a38-170f-486f-83d5-7bc427700f93 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:04:36.521920 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 3/4] 10.222.0.23 () {58 vars in 1324 bytes} [Tue Jun 21 18:04:36 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 18:04:36.528034 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7b400f-032d-40ce-86d2-464ffece70c7 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:36.542878 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7b400f-032d-40ce-86d2-464ffece70c7 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:36.547366 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095f72fb-3460-4b52-bbcf-18a5500fc49c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:36.548551 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f72fb-3460-4b52-bbcf-18a5500fc49c admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:36.548840 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f72fb-3460-4b52-bbcf-18a5500fc49c admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:36.549113 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f72fb-3460-4b52-bbcf-18a5500fc49c admin admin] PATH_INFO: `/v3/projects/cecad7a8a62644cabfc30fc28ce80d90` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:36.553255 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095f72fb-3460-4b52-bbcf-18a5500fc49c admin admin] RBAC: Authorizing `identity:get_project(project_id=cecad7a8a62644cabfc30fc28ce80d90)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:04:36.605573 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: WARNING keystone.common.rbac_enforcer.enforcer [None req-095f72fb-3460-4b52-bbcf-18a5500fc49c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 21 18:04:36.605910 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095f72fb-3460-4b52-bbcf-18a5500fc49c admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:04:36.611116 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 2/5] 10.222.0.23 () {56 vars in 1207 bytes} [Tue Jun 21 18:04:36 2022] GET /identity/v3/projects/cecad7a8a62644cabfc30fc28ce80d90 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:04:38.629571 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac749f8d-b587-419c-911e-4431983a3600 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:38.630108 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac749f8d-b587-419c-911e-4431983a3600 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:38.630108 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac749f8d-b587-419c-911e-4431983a3600 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:38.634199 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 4/6] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:04:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:04:38.640087 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d84a468-3d76-4b13-a4eb-d529604dc3c4 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:38.640561 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d84a468-3d76-4b13-a4eb-d529604dc3c4 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:38.640716 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d84a468-3d76-4b13-a4eb-d529604dc3c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:38.659664 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-3d84a468-3d76-4b13-a4eb-d529604dc3c4 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:04:38.661259 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-3d84a468-3d76-4b13-a4eb-d529604dc3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:04:38.670198 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 3/7] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:04:38 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 21 18:04:38.693543 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac749f8d-b587-419c-911e-4431983a3600 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:38.706210 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac749f8d-b587-419c-911e-4431983a3600 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:38.710492 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7919754-7693-4553-b2d2-a76f997af5c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:38.711142 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7919754-7693-4553-b2d2-a76f997af5c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:38.711431 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7919754-7693-4553-b2d2-a76f997af5c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:38.711711 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7919754-7693-4553-b2d2-a76f997af5c2 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:38.720654 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7919754-7693-4553-b2d2-a76f997af5c2 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:04:38.722717 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7919754-7693-4553-b2d2-a76f997af5c2 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:04:38.732669 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 5/8] 10.222.0.23 () {58 vars in 1324 bytes} [Tue Jun 21 18:04:38 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 21 18:04:38.738544 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d84a468-3d76-4b13-a4eb-d529604dc3c4 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:38.748215 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d84a468-3d76-4b13-a4eb-d529604dc3c4 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:38.752688 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea326aa0-4daa-42f9-b4f1-1b876f26099c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:38.753564 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea326aa0-4daa-42f9-b4f1-1b876f26099c admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:38.753846 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea326aa0-4daa-42f9-b4f1-1b876f26099c admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:38.754121 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea326aa0-4daa-42f9-b4f1-1b876f26099c admin admin] PATH_INFO: `/v3/users/0c89ac265bd34c75a692d91d672fad59` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:38.755514 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea326aa0-4daa-42f9-b4f1-1b876f26099c admin admin] RBAC: Authorizing `identity:get_user(user_id=0c89ac265bd34c75a692d91d672fad59)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:04:38.757370 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea326aa0-4daa-42f9-b4f1-1b876f26099c admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:04:38.758960 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 4/9] 10.222.0.23 () {56 vars in 1198 bytes} [Tue Jun 21 18:04:38 2022] GET /identity/v3/users/0c89ac265bd34c75a692d91d672fad59 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:04:40.918304 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3b661-4131-4219-abfe-cf8fddd9aa24 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:40.918883 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3b661-4131-4219-abfe-cf8fddd9aa24 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:40.918883 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3b661-4131-4219-abfe-cf8fddd9aa24 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:40.920257 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 6/10] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:04:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:04:40.925775 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b7991-bf52-4a9e-8bb0-185a130734ef None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:40.926264 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b7991-bf52-4a9e-8bb0-185a130734ef None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:40.926384 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b7991-bf52-4a9e-8bb0-185a130734ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:40.943537 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-316b7991-bf52-4a9e-8bb0-185a130734ef None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:04:40.944804 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-316b7991-bf52-4a9e-8bb0-185a130734ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:04:40.986760 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576be03-13d7-4627-8234-94173268de91 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:40.987050 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576be03-13d7-4627-8234-94173268de91 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:40.987339 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576be03-13d7-4627-8234-94173268de91 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:40.988812 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 7/11] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:04:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:04:40.995920 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6efc7be-5f46-465c-a248-453597690f55 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:40.996289 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6efc7be-5f46-465c-a248-453597690f55 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:40.996700 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6efc7be-5f46-465c-a248-453597690f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:41.043218 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG passlib.handlers.bcrypt [None req-f6efc7be-5f46-465c-a248-453597690f55 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80614) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 21 18:04:41.043915 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG passlib.handlers.bcrypt [None req-f6efc7be-5f46-465c-a248-453597690f55 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80614) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 21 18:04:41.048488 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 5/12] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:04:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:04:41.073098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316b7991-bf52-4a9e-8bb0-185a130734ef None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:41.091842 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-f6efc7be-5f46-465c-a248-453597690f55 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:04:41.093508 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-f6efc7be-5f46-465c-a248-453597690f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:04:41.103012 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:04:41.193286 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316b7991-bf52-4a9e-8bb0-185a130734ef None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:41.202637 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 8/13] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:04:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:04:41.225238 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6efc7be-5f46-465c-a248-453597690f55 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:41.230241 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1b2ca4-093c-47c3-b1a9-5f1fcb4b8b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:41.231054 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b2ca4-093c-47c3-b1a9-5f1fcb4b8b1e None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:41.231339 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b2ca4-093c-47c3-b1a9-5f1fcb4b8b1e None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:41.231614 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b2ca4-093c-47c3-b1a9-5f1fcb4b8b1e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:41.243762 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1b2ca4-093c-47c3-b1a9-5f1fcb4b8b1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:04:41.256480 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-ca1b2ca4-093c-47c3-b1a9-5f1fcb4b8b1e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:04:41.256480 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:04:41.256480 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:04:41.256480 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:04:41.256480 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:04:41.256480 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:04:41.256480 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:04:41.256480 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:04:41.256480 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:04:41.256480 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:04:41.256480 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:04:41.256480 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:04:41.256480 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:04:41.256480 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:04:41.256480 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:04:41.256480 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:04:41.258209 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:04:41.258209 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:04:41.258209 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:04:41.258209 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:04:41.258209 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:04:41.258209 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:04:41.258209 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:04:41.258209 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:04:41.258209 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:04:41.258209 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:04:41.258209 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:04:41.258209 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:04:41.258209 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:04:41.258209 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:04:41.258209 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:04:41.258209 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:04:41.259880 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:04:41.259880 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:04:41.259880 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:04:41.259880 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:04:41.259880 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:04:41.259880 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:04:41.259880 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:04:41.259880 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:04:41.259880 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:04:41.259880 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:04:41.259880 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:04:41.259880 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:04:41.259880 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 6/14] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jun 21 18:04:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:04:41.264721 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1b2ca4-093c-47c3-b1a9-5f1fcb4b8b1e None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:41.344858 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6efc7be-5f46-465c-a248-453597690f55 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:41.347615 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:04:41.358858 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:04:41.378093 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1b2ca4-093c-47c3-b1a9-5f1fcb4b8b1e None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:41.383983 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb67f334-1a08-4ac9-b9e6-f27f6c7acf8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:41.384997 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67f334-1a08-4ac9-b9e6-f27f6c7acf8c None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:41.385290 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67f334-1a08-4ac9-b9e6-f27f6c7acf8c None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:41.385841 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67f334-1a08-4ac9-b9e6-f27f6c7acf8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:41.415114 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9261cb40-216d-448f-9704-32ac4701b7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:41.415802 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9261cb40-216d-448f-9704-32ac4701b7e2 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:41.416088 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9261cb40-216d-448f-9704-32ac4701b7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:41.416369 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9261cb40-216d-448f-9704-32ac4701b7e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:41.417687 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9261cb40-216d-448f-9704-32ac4701b7e2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:04:41.418271 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9261cb40-216d-448f-9704-32ac4701b7e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:04:41.419416 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9261cb40-216d-448f-9704-32ac4701b7e2 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:04:41.432253 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 7/15] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:04:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:04:41.439446 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9261cb40-216d-448f-9704-32ac4701b7e2 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:41.457485 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb67f334-1a08-4ac9-b9e6-f27f6c7acf8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:04:41.458952 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb67f334-1a08-4ac9-b9e6-f27f6c7acf8c None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:04:41.560937 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9261cb40-216d-448f-9704-32ac4701b7e2 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:41.582305 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 9/16] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:04:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 21 18:04:41.588426 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb67f334-1a08-4ac9-b9e6-f27f6c7acf8c None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:41.592742 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:04:41.599137 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017c7606-8ab4-4e32-8146-0dfe58f30136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:41.600031 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017c7606-8ab4-4e32-8146-0dfe58f30136 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:41.606403 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017c7606-8ab4-4e32-8146-0dfe58f30136 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:41.606403 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017c7606-8ab4-4e32-8146-0dfe58f30136 None admin] PATH_INFO: `/v3/users/0c89ac265bd34c75a692d91d672fad59` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:41.606403 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017c7606-8ab4-4e32-8146-0dfe58f30136 None admin] RBAC: Authorizing `identity:get_user(user_id=0c89ac265bd34c75a692d91d672fad59)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:04:41.606403 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017c7606-8ab4-4e32-8146-0dfe58f30136 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:04:41.606403 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 8/17] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jun 21 18:04:41 2022] GET /identity/v3/users/0c89ac265bd34c75a692d91d672fad59 => generated 247 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:04:41.610548 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017c7606-8ab4-4e32-8146-0dfe58f30136 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:41.626798 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:04:41.696932 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb67f334-1a08-4ac9-b9e6-f27f6c7acf8c None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:41.717597 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017c7606-8ab4-4e32-8146-0dfe58f30136 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:41.733502 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509b3a44-14a9-4711-923c-a5ad3d3f0526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:41.734308 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b3a44-14a9-4711-923c-a5ad3d3f0526 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:41.734606 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b3a44-14a9-4711-923c-a5ad3d3f0526 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:41.734886 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b3a44-14a9-4711-923c-a5ad3d3f0526 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:41.752264 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec50a2cd-c4c9-422b-83f5-8b4c85d2756f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:41.753092 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50a2cd-c4c9-422b-83f5-8b4c85d2756f None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:41.753380 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50a2cd-c4c9-422b-83f5-8b4c85d2756f None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:41.753664 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50a2cd-c4c9-422b-83f5-8b4c85d2756f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:41.754880 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec50a2cd-c4c9-422b-83f5-8b4c85d2756f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:04:41.756088 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509b3a44-14a9-4711-923c-a5ad3d3f0526 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:04:41.756348 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec50a2cd-c4c9-422b-83f5-8b4c85d2756f None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:04:41.757592 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509b3a44-14a9-4711-923c-a5ad3d3f0526 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:04:41.758305 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 9/18] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:04:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:04:41.764493 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec50a2cd-c4c9-422b-83f5-8b4c85d2756f None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:41.780564 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-509b3a44-14a9-4711-923c-a5ad3d3f0526 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:04:41.780564 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:04:41.780564 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:04:41.780564 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:04:41.780564 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:04:41.780564 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:04:41.780564 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:04:41.780564 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:04:41.780564 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:04:41.780564 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:04:41.780564 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:04:41.780564 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:04:41.780564 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:04:41.780564 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:04:41.780564 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:04:41.781635 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:04:41.781635 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:04:41.781635 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:04:41.781635 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:04:41.781635 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:04:41.781635 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:04:41.781635 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:04:41.781635 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:04:41.781635 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:04:41.781635 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:04:41.781635 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:04:41.781635 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:04:41.781635 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:04:41.781635 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:04:41.781635 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:04:41.781635 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:04:41.781635 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:04:41.783444 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:04:41.783444 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:04:41.783444 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:04:41.783444 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:04:41.783444 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 10/19] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:04:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:04:41.788981 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509b3a44-14a9-4711-923c-a5ad3d3f0526 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:41.848223 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:04:41.861431 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:04:41.881741 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec50a2cd-c4c9-422b-83f5-8b4c85d2756f None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:41.899962 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509b3a44-14a9-4711-923c-a5ad3d3f0526 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:41.923759 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c6417b-2fa8-409b-964b-a5616491a861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:41.924665 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6417b-2fa8-409b-964b-a5616491a861 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:41.925032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6417b-2fa8-409b-964b-a5616491a861 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:41.925365 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6417b-2fa8-409b-964b-a5616491a861 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:41.926392 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c6417b-2fa8-409b-964b-a5616491a861 None admin] RBAC: Adding query filter params (role.id=745ed9065a8143e384ebbd694b29635c, scope.domain.id=default, user.id=0c89ac265bd34c75a692d91d672fad59) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:04:41.926976 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c6417b-2fa8-409b-964b-a5616491a861 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:04:41.928622 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c6417b-2fa8-409b-964b-a5616491a861 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:04:41.937689 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 10/20] 10.222.0.23 () {56 vars in 1322 bytes} [Tue Jun 21 18:04:41 2022] GET /identity/v3/role_assignments?user.id=0c89ac265bd34c75a692d91d672fad59&scope.domain.id=default&role.id=745ed9065a8143e384ebbd694b29635c => generated 233 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:04:41.938328 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02166f1-988a-44ed-bc9d-6fa53bc25b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:41.939003 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02166f1-988a-44ed-bc9d-6fa53bc25b3d None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:41.939291 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02166f1-988a-44ed-bc9d-6fa53bc25b3d None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:41.939569 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02166f1-988a-44ed-bc9d-6fa53bc25b3d None admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:41.940317 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02166f1-988a-44ed-bc9d-6fa53bc25b3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:04:41.940663 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02166f1-988a-44ed-bc9d-6fa53bc25b3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:04:41.942101 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02166f1-988a-44ed-bc9d-6fa53bc25b3d None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:04:41.963466 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 11/21] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:04:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:04:44.887110 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba04984-79cf-48be-8650-6fced39eee00 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:44.887518 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba04984-79cf-48be-8650-6fced39eee00 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:44.887671 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba04984-79cf-48be-8650-6fced39eee00 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:44.889265 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 11/22] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:04:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:04:44.895379 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84abaad7-a771-4336-beb8-7ce10efaafb3 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:44.896188 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84abaad7-a771-4336-beb8-7ce10efaafb3 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:44.896651 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84abaad7-a771-4336-beb8-7ce10efaafb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:44.919337 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-84abaad7-a771-4336-beb8-7ce10efaafb3 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:04:44.920819 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-84abaad7-a771-4336-beb8-7ce10efaafb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:04:44.934161 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b719b4e-ce2e-4d9a-96b4-6b69b43f4cfd None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:44.934781 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b719b4e-ce2e-4d9a-96b4-6b69b43f4cfd None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:44.935305 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b719b4e-ce2e-4d9a-96b4-6b69b43f4cfd None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:44.939995 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 12/23] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:04:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:04:44.944568 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e47da-c7b3-4d2e-897a-a9c2cb0877db None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:44.945183 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e47da-c7b3-4d2e-897a-a9c2cb0877db None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:44.945667 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e47da-c7b3-4d2e-897a-a9c2cb0877db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:44.985983 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-380e47da-c7b3-4d2e-897a-a9c2cb0877db None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:04:44.987695 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-380e47da-c7b3-4d2e-897a-a9c2cb0877db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:04:45.080561 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 12/24] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:04:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:04:45.091368 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 13/25] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:04:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 21 18:04:45.127287 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380e47da-c7b3-4d2e-897a-a9c2cb0877db None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:45.127464 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84abaad7-a771-4336-beb8-7ce10efaafb3 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:45.230256 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:04:45.241415 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:04:45.268980 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380e47da-c7b3-4d2e-897a-a9c2cb0877db None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:45.312655 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e26113-61a7-4036-a43b-31a19d55a911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:45.313471 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e26113-61a7-4036-a43b-31a19d55a911 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:45.313797 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e26113-61a7-4036-a43b-31a19d55a911 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:45.314072 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e26113-61a7-4036-a43b-31a19d55a911 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:45.326906 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e26113-61a7-4036-a43b-31a19d55a911 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:04:45.328426 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e26113-61a7-4036-a43b-31a19d55a911 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:04:45.350654 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-d2e26113-61a7-4036-a43b-31a19d55a911 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:04:45.350654 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:04:45.350654 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:04:45.350654 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:04:45.350654 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:04:45.350654 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:04:45.350654 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:04:45.350654 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:04:45.350654 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:04:45.350654 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:04:45.350654 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:04:45.350654 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:04:45.350654 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:04:45.350654 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:04:45.350654 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:04:45.351545 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:04:45.351545 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:04:45.351545 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:04:45.351545 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:04:45.351545 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:04:45.351545 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:04:45.351545 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:04:45.351545 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:04:45.351545 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:04:45.351545 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:04:45.351545 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:04:45.351545 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:04:45.351545 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:04:45.351545 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:04:45.351545 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:04:45.351545 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:04:45.351545 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:04:45.352355 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:04:45.352355 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:04:45.352355 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:04:45.352355 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:04:45.352355 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 14/26] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:04:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:04:45.353411 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1991bb-3698-4a2c-b8ee-dd4f6e7d2cf9 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:45.353577 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1991bb-3698-4a2c-b8ee-dd4f6e7d2cf9 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:45.356423 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1991bb-3698-4a2c-b8ee-dd4f6e7d2cf9 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:45.360482 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 15/27] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:04:45 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:04:45.361399 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1991bb-3698-4a2c-b8ee-dd4f6e7d2cf9 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:45.405811 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84abaad7-a771-4336-beb8-7ce10efaafb3 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:45.487205 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1991bb-3698-4a2c-b8ee-dd4f6e7d2cf9 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:45.491370 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188278f8-a419-439c-85ac-ac584a18e803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:45.493210 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188278f8-a419-439c-85ac-ac584a18e803 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:45.493944 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188278f8-a419-439c-85ac-ac584a18e803 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:45.494616 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188278f8-a419-439c-85ac-ac584a18e803 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:45.517674 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188278f8-a419-439c-85ac-ac584a18e803 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:04:45.517986 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:04:45.528320 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5086b0-4e56-4e32-850e-f6c6b53a1999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:45.529031 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5086b0-4e56-4e32-850e-f6c6b53a1999 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:45.529314 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5086b0-4e56-4e32-850e-f6c6b53a1999 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:45.529588 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5086b0-4e56-4e32-850e-f6c6b53a1999 None admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:45.534307 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5086b0-4e56-4e32-850e-f6c6b53a1999 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:04:45.534307 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5086b0-4e56-4e32-850e-f6c6b53a1999 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:04:45.534307 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5086b0-4e56-4e32-850e-f6c6b53a1999 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:04:45.539451 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-188278f8-a419-439c-85ac-ac584a18e803 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:04:45.539451 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:04:45.539451 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:04:45.539451 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:04:45.539451 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:04:45.539451 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:04:45.539451 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:04:45.539451 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:04:45.539451 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:04:45.539451 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:04:45.539451 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:04:45.539451 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:04:45.539451 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:04:45.539451 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:04:45.539451 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:04:45.539451 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:04:45.541474 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:04:45.541474 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:04:45.541474 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:04:45.541474 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:04:45.541474 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:04:45.541474 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:04:45.541474 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:04:45.541474 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:04:45.541474 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:04:45.541474 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:04:45.541474 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:04:45.541474 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:04:45.541474 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:04:45.541474 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:04:45.541474 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:04:45.541474 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:04:45.544355 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:04:45.544355 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:04:45.544355 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:04:45.544355 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:04:45.544355 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:04:45.544355 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:04:45.544355 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:04:45.544355 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:04:45.544355 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:04:45.544355 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:04:45.544355 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:04:45.544355 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:04:45.548262 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 13/28] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jun 21 18:04:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 422 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:04:45.549920 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c1c1f6-354e-407b-914a-9e01537c6d9d None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:45.553116 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c1c1f6-354e-407b-914a-9e01537c6d9d None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:45.553116 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c1c1f6-354e-407b-914a-9e01537c6d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:45.558720 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 16/29] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:04:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:04:45.560548 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df24a8-945c-4753-b402-e7c583c9999a None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:45.561050 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df24a8-945c-4753-b402-e7c583c9999a None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:45.561500 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df24a8-945c-4753-b402-e7c583c9999a None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:45.563437 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 17/30] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:04:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:04:45.564565 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01df24a8-945c-4753-b402-e7c583c9999a None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:45.578600 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-d5c1c1f6-354e-407b-914a-9e01537c6d9d None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:04:45.580466 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-d5c1c1f6-354e-407b-914a-9e01537c6d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:04:45.680501 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 14/31] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:04:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:04:45.681639 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c1c1f6-354e-407b-914a-9e01537c6d9d None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:45.725946 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01df24a8-945c-4753-b402-e7c583c9999a None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:45.733273 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:04:45.774993 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee5555c-173e-4fec-9da9-7af20c146a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:45.775758 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee5555c-173e-4fec-9da9-7af20c146a0c None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:45.776787 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee5555c-173e-4fec-9da9-7af20c146a0c None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:45.777239 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee5555c-173e-4fec-9da9-7af20c146a0c None admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:45.779115 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee5555c-173e-4fec-9da9-7af20c146a0c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:04:45.779672 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee5555c-173e-4fec-9da9-7af20c146a0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:04:45.781325 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee5555c-173e-4fec-9da9-7af20c146a0c None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:04:45.794699 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 18/32] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:04:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:04:45.796771 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d1f89b-3e39-40f8-8fb8-c99b586e52aa None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:45.797229 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d1f89b-3e39-40f8-8fb8-c99b586e52aa None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:45.797681 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d1f89b-3e39-40f8-8fb8-c99b586e52aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:45.813824 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c1c1f6-354e-407b-914a-9e01537c6d9d None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:45.820947 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-80d1f89b-3e39-40f8-8fb8-c99b586e52aa None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:04:45.823518 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-80d1f89b-3e39-40f8-8fb8-c99b586e52aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:04:45.860382 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2109391e-a23b-4778-8277-20bbe5ddac49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:45.861461 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2109391e-a23b-4778-8277-20bbe5ddac49 None admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:45.862073 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2109391e-a23b-4778-8277-20bbe5ddac49 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:45.862657 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2109391e-a23b-4778-8277-20bbe5ddac49 None admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:45.863827 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2109391e-a23b-4778-8277-20bbe5ddac49 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:04:45.865572 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2109391e-a23b-4778-8277-20bbe5ddac49 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:04:45.880654 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:04:45.910297 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 19/33] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:04:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:04:45.912673 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d1f89b-3e39-40f8-8fb8-c99b586e52aa None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:46.037531 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d1f89b-3e39-40f8-8fb8-c99b586e52aa None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:46.078830 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c036190-2c87-4113-8ae1-36139f7c3e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:46.079762 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c036190-2c87-4113-8ae1-36139f7c3e96 None admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:46.080099 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c036190-2c87-4113-8ae1-36139f7c3e96 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:46.080400 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c036190-2c87-4113-8ae1-36139f7c3e96 None admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:46.081186 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c036190-2c87-4113-8ae1-36139f7c3e96 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:04:46.082651 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c036190-2c87-4113-8ae1-36139f7c3e96 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:04:46.123293 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 15/34] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Jun 21 18:04:45 2022] POST /identity/v3/projects => generated 301 bytes in 442 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:04:46.125035 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2109391e-a23b-4778-8277-20bbe5ddac49 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:46.240845 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:04:46.252995 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2109391e-a23b-4778-8277-20bbe5ddac49 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:46.295103 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060da1e2-e873-433a-9e0c-4ec3d703f85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:46.296374 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060da1e2-e873-433a-9e0c-4ec3d703f85e None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:46.296917 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060da1e2-e873-433a-9e0c-4ec3d703f85e None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:46.297412 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060da1e2-e873-433a-9e0c-4ec3d703f85e None admin] PATH_INFO: `/v3/users/0c89ac265bd34c75a692d91d672fad59` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:46.299468 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060da1e2-e873-433a-9e0c-4ec3d703f85e None admin] RBAC: Authorizing `identity:get_user(user_id=0c89ac265bd34c75a692d91d672fad59)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:04:46.301548 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060da1e2-e873-433a-9e0c-4ec3d703f85e None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:04:46.303722 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 16/35] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jun 21 18:04:46 2022] GET /identity/v3/users/0c89ac265bd34c75a692d91d672fad59 => generated 247 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:04:46.304968 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060da1e2-e873-433a-9e0c-4ec3d703f85e None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:46.432984 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060da1e2-e873-433a-9e0c-4ec3d703f85e None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:46.456508 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 20/36] 10.222.0.23 () {60 vars in 1131 bytes} [Tue Jun 21 18:04:45 2022] POST /identity/v3/roles => generated 225 bytes in 536 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:04:46.458503 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a25b8-b752-47bb-be34-651aab335b18 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:46.458807 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a25b8-b752-47bb-be34-651aab335b18 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:46.459095 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a25b8-b752-47bb-be34-651aab335b18 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:46.460575 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 21/37] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:04:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:04:46.461465 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837a25b8-b752-47bb-be34-651aab335b18 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:46.481427 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6dd468-775e-4c29-8aa7-545e956bd561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:46.482133 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6dd468-775e-4c29-8aa7-545e956bd561 None admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:46.482427 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6dd468-775e-4c29-8aa7-545e956bd561 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:46.482714 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6dd468-775e-4c29-8aa7-545e956bd561 None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:46.483503 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6dd468-775e-4c29-8aa7-545e956bd561 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:04:46.485038 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6dd468-775e-4c29-8aa7-545e956bd561 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:04:46.516105 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:04:46.557747 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 17/38] 10.222.0.23 () {60 vars in 1131 bytes} [Tue Jun 21 18:04:46 2022] POST /identity/v3/roles => generated 219 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:04:46.559259 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f1e13d-42db-4c3b-8ee6-86fac25f1283 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:46.559606 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f1e13d-42db-4c3b-8ee6-86fac25f1283 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:46.559898 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f1e13d-42db-4c3b-8ee6-86fac25f1283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:46.584856 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-76f1e13d-42db-4c3b-8ee6-86fac25f1283 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:04:46.587679 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-76f1e13d-42db-4c3b-8ee6-86fac25f1283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:04:46.609146 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:04:46.634399 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837a25b8-b752-47bb-be34-651aab335b18 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:46.687542 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 18/39] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:04:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:04:46.695632 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2be8716-1314-442a-9232-79d39d09d26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:46.696167 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be8716-1314-442a-9232-79d39d09d26b None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:46.697346 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be8716-1314-442a-9232-79d39d09d26b None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:46.697346 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be8716-1314-442a-9232-79d39d09d26b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:46.698675 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2be8716-1314-442a-9232-79d39d09d26b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:04:46.700321 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2be8716-1314-442a-9232-79d39d09d26b None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:04:46.702244 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 22/40] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:04:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:04:46.707807 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f1e13d-42db-4c3b-8ee6-86fac25f1283 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:46.721166 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2be8716-1314-442a-9232-79d39d09d26b None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:46.833820 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f1e13d-42db-4c3b-8ee6-86fac25f1283 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:46.846078 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2be8716-1314-442a-9232-79d39d09d26b None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:46.854591 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:04:46.873909 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0959e5-0630-45d7-9118-1d3dac4a9468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:46.874819 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0959e5-0630-45d7-9118-1d3dac4a9468 None admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:46.875114 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0959e5-0630-45d7-9118-1d3dac4a9468 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:46.875400 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0959e5-0630-45d7-9118-1d3dac4a9468 None admin] PATH_INFO: `/v3/domains/default/users/0c89ac265bd34c75a692d91d672fad59/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:46.877502 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0959e5-0630-45d7-9118-1d3dac4a9468 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=0c89ac265bd34c75a692d91d672fad59, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:04:46.878964 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0959e5-0630-45d7-9118-1d3dac4a9468 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:04:46.881670 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0347bf-40e2-422f-8ced-626444e118b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:46.882366 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0347bf-40e2-422f-8ced-626444e118b0 None admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:46.882655 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0347bf-40e2-422f-8ced-626444e118b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:46.882937 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0347bf-40e2-422f-8ced-626444e118b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:46.883706 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0347bf-40e2-422f-8ced-626444e118b0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:04:46.885143 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0347bf-40e2-422f-8ced-626444e118b0 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:04:46.976133 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 19/41] 10.222.0.23 () {58 vars in 1361 bytes} [Tue Jun 21 18:04:46 2022] PUT /identity/v3/domains/default/users/0c89ac265bd34c75a692d91d672fad59/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:04:46.977562 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff73d40e-4bbb-46bd-9bcc-5aa5bfacf828 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:46.977884 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff73d40e-4bbb-46bd-9bcc-5aa5bfacf828 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:46.978169 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff73d40e-4bbb-46bd-9bcc-5aa5bfacf828 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:46.979746 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 20/42] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:04:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:04:46.985669 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a90a8a-bd2b-41bf-a64f-4dac07482315 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:46.986096 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a90a8a-bd2b-41bf-a64f-4dac07482315 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:46.986246 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a90a8a-bd2b-41bf-a64f-4dac07482315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:46.999476 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:04:47.002202 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 23/43] 10.222.0.23 () {60 vars in 1131 bytes} [Tue Jun 21 18:04:46 2022] POST /identity/v3/roles => generated 223 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:04:47.006089 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-d9a90a8a-bd2b-41bf-a64f-4dac07482315 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:04:47.007394 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-d9a90a8a-bd2b-41bf-a64f-4dac07482315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:04:47.093607 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 21/44] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:04:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:04:47.118116 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0347bf-40e2-422f-8ced-626444e118b0 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:47.247124 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0347bf-40e2-422f-8ced-626444e118b0 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:47.299499 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e269fc2-f759-4cb7-9d83-d7696ced3a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:47.300917 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e269fc2-f759-4cb7-9d83-d7696ced3a31 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:47.301436 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e269fc2-f759-4cb7-9d83-d7696ced3a31 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:47.301943 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e269fc2-f759-4cb7-9d83-d7696ced3a31 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:47.304007 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e269fc2-f759-4cb7-9d83-d7696ced3a31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:04:47.306816 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e269fc2-f759-4cb7-9d83-d7696ced3a31 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:04:47.315345 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 24/45] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:04:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:04:47.316524 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a90a8a-bd2b-41bf-a64f-4dac07482315 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:47.418275 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a90a8a-bd2b-41bf-a64f-4dac07482315 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:47.452650 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77464d84-789d-4fa3-9189-5495feff6ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:47.453320 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77464d84-789d-4fa3-9189-5495feff6ed7 None admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:47.453606 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77464d84-789d-4fa3-9189-5495feff6ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:47.453884 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77464d84-789d-4fa3-9189-5495feff6ed7 None admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:47.454608 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77464d84-789d-4fa3-9189-5495feff6ed7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:04:47.456058 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77464d84-789d-4fa3-9189-5495feff6ed7 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:04:47.460424 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:04:47.552428 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 22/46] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Jun 21 18:04:47 2022] POST /identity/v3/projects => generated 312 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:04:49.122649 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d470247-0b16-49c3-b417-75da0439a58c None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:49.123199 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d470247-0b16-49c3-b417-75da0439a58c None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:49.123199 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d470247-0b16-49c3-b417-75da0439a58c None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:49.124618 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 25/47] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:04:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:04:49.130076 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f71566-69c2-42d2-a13b-999172aa0058 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:49.130543 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f71566-69c2-42d2-a13b-999172aa0058 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:49.130674 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f71566-69c2-42d2-a13b-999172aa0058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:49.148691 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-72f71566-69c2-42d2-a13b-999172aa0058 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:04:49.149829 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-72f71566-69c2-42d2-a13b-999172aa0058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:04:49.226963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 23/48] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:04:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:04:49.252876 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d470247-0b16-49c3-b417-75da0439a58c None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:49.293887 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:04:49.366939 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d470247-0b16-49c3-b417-75da0439a58c None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:49.403534 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a804c86a-f695-4a24-a58f-d9c732cd65d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:49.404344 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a804c86a-f695-4a24-a58f-d9c732cd65d9 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:49.404664 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a804c86a-f695-4a24-a58f-d9c732cd65d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:49.404949 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a804c86a-f695-4a24-a58f-d9c732cd65d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:49.415570 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a804c86a-f695-4a24-a58f-d9c732cd65d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:04:49.423429 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-a804c86a-f695-4a24-a58f-d9c732cd65d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:04:49.423429 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:04:49.423429 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:04:49.423429 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:04:49.423429 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:04:49.423429 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:04:49.423429 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:04:49.423429 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:04:49.423429 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:04:49.423429 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:04:49.423429 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:04:49.423429 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:04:49.423429 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:04:49.423429 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:04:49.423429 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:04:49.423429 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:04:49.424433 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:04:49.424433 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:04:49.424433 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:04:49.424433 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:04:49.424433 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:04:49.424433 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:04:49.424433 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:04:49.424433 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:04:49.424433 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:04:49.424433 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:04:49.424433 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:04:49.424433 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:04:49.424433 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:04:49.424433 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:04:49.424433 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:04:49.424433 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:04:49.425505 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:04:49.425505 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:04:49.425505 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:04:49.425505 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:04:49.425505 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:04:49.425505 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:04:49.425505 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:04:49.425505 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:04:49.425505 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:04:49.425505 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:04:49.425505 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:04:49.425505 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:04:49.425505 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 26/49] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jun 21 18:04:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:04:49.430574 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f71566-69c2-42d2-a13b-999172aa0058 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:49.532698 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f71566-69c2-42d2-a13b-999172aa0058 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:49.550630 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:04:49.569975 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465e6d19-4e11-4531-96d1-fabdcbb1ba15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:49.570673 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e6d19-4e11-4531-96d1-fabdcbb1ba15 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:49.570991 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e6d19-4e11-4531-96d1-fabdcbb1ba15 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:49.571277 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e6d19-4e11-4531-96d1-fabdcbb1ba15 None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:49.572051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465e6d19-4e11-4531-96d1-fabdcbb1ba15 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:04:49.574887 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465e6d19-4e11-4531-96d1-fabdcbb1ba15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:04:49.574887 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465e6d19-4e11-4531-96d1-fabdcbb1ba15 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:04:49.587914 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 24/50] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:04:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:04:49.594131 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a804c86a-f695-4a24-a58f-d9c732cd65d9 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:49.619757 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1169ccd-f105-4a2e-88a3-48a3e600c5a3 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:49.620085 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1169ccd-f105-4a2e-88a3-48a3e600c5a3 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:49.620429 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1169ccd-f105-4a2e-88a3-48a3e600c5a3 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:49.621983 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 25/51] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:04:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:04:49.627132 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be9586-d522-4abd-98e6-1d4274c3cb93 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:49.627422 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be9586-d522-4abd-98e6-1d4274c3cb93 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:49.627701 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be9586-d522-4abd-98e6-1d4274c3cb93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:49.645651 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-76be9586-d522-4abd-98e6-1d4274c3cb93 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:04:49.646879 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-76be9586-d522-4abd-98e6-1d4274c3cb93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:04:49.692576 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:04:49.699519 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a804c86a-f695-4a24-a58f-d9c732cd65d9 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:49.719010 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 26/52] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:04:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:04:49.735083 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ab6e11-7a8d-4751-83a8-327ab5833848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:49.735965 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ab6e11-7a8d-4751-83a8-327ab5833848 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:49.736250 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ab6e11-7a8d-4751-83a8-327ab5833848 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:49.736557 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ab6e11-7a8d-4751-83a8-327ab5833848 None admin] PATH_INFO: `/v3/users/0c89ac265bd34c75a692d91d672fad59` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:49.737762 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ab6e11-7a8d-4751-83a8-327ab5833848 None admin] RBAC: Authorizing `identity:get_user(user_id=0c89ac265bd34c75a692d91d672fad59)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:04:49.739212 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ab6e11-7a8d-4751-83a8-327ab5833848 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:04:49.740894 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 27/53] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jun 21 18:04:49 2022] GET /identity/v3/users/0c89ac265bd34c75a692d91d672fad59 => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:04:49.742354 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76be9586-d522-4abd-98e6-1d4274c3cb93 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:49.746331 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ab6e11-7a8d-4751-83a8-327ab5833848 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:49.862712 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ab6e11-7a8d-4751-83a8-327ab5833848 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:49.906505 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad0847a-e7a3-4ca0-a89e-49e33ebca318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:49.907323 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0847a-e7a3-4ca0-a89e-49e33ebca318 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:49.907611 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0847a-e7a3-4ca0-a89e-49e33ebca318 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:49.907891 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0847a-e7a3-4ca0-a89e-49e33ebca318 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:49.909167 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad0847a-e7a3-4ca0-a89e-49e33ebca318 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:04:49.910651 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad0847a-e7a3-4ca0-a89e-49e33ebca318 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:04:49.912311 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 28/54] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:04:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:04:49.918843 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad0847a-e7a3-4ca0-a89e-49e33ebca318 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:49.952383 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:04:49.958832 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76be9586-d522-4abd-98e6-1d4274c3cb93 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:49.971972 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:04:49.999184 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b6d989-c6ee-4f93-aafc-f642dcbfcea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:50.000102 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6d989-c6ee-4f93-aafc-f642dcbfcea3 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:50.000475 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6d989-c6ee-4f93-aafc-f642dcbfcea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:50.000817 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6d989-c6ee-4f93-aafc-f642dcbfcea3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:50.002141 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b6d989-c6ee-4f93-aafc-f642dcbfcea3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:04:50.003845 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b6d989-c6ee-4f93-aafc-f642dcbfcea3 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:04:50.005709 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 27/55] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:04:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:04:50.012683 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b6d989-c6ee-4f93-aafc-f642dcbfcea3 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:50.032926 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad0847a-e7a3-4ca0-a89e-49e33ebca318 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:50.072041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6789b60-abf4-4293-b240-cb14dc0e77c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:50.072737 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6789b60-abf4-4293-b240-cb14dc0e77c9 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:50.073612 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6789b60-abf4-4293-b240-cb14dc0e77c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:50.074166 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6789b60-abf4-4293-b240-cb14dc0e77c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:50.074797 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6789b60-abf4-4293-b240-cb14dc0e77c9 None admin] RBAC: Adding query filter params (role.id=745ed9065a8143e384ebbd694b29635c, scope.domain.id=default, user.id=0c89ac265bd34c75a692d91d672fad59) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:04:50.075112 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6789b60-abf4-4293-b240-cb14dc0e77c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:04:50.076602 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6789b60-abf4-4293-b240-cb14dc0e77c9 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:04:50.082500 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 29/56] 10.222.0.23 () {56 vars in 1322 bytes} [Tue Jun 21 18:04:49 2022] GET /identity/v3/role_assignments?user.id=0c89ac265bd34c75a692d91d672fad59&scope.domain.id=default&role.id=745ed9065a8143e384ebbd694b29635c => generated 530 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:04:50.125286 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b6d989-c6ee-4f93-aafc-f642dcbfcea3 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:50.163051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791b6fcc-48d5-4ab4-a632-3ba77f834f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:50.163748 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791b6fcc-48d5-4ab4-a632-3ba77f834f95 None admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:50.164015 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791b6fcc-48d5-4ab4-a632-3ba77f834f95 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:50.164301 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791b6fcc-48d5-4ab4-a632-3ba77f834f95 None admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:50.165077 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791b6fcc-48d5-4ab4-a632-3ba77f834f95 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:04:50.166574 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791b6fcc-48d5-4ab4-a632-3ba77f834f95 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:04:50.253016 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 28/57] 10.222.0.23 () {60 vars in 1140 bytes} [Tue Jun 21 18:04:50 2022] POST /identity/v3/projects => generated 298 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:04:52.312338 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da42ceba-5c1c-4602-82c8-ea6ffbdadbe1 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:52.312878 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da42ceba-5c1c-4602-82c8-ea6ffbdadbe1 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:52.312989 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da42ceba-5c1c-4602-82c8-ea6ffbdadbe1 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:52.314313 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 30/58] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:04:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:04:52.319369 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8200c33f-065c-4b63-8f9c-1b90a26fba71 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:52.319816 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8200c33f-065c-4b63-8f9c-1b90a26fba71 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:52.319946 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8200c33f-065c-4b63-8f9c-1b90a26fba71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:52.337100 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-8200c33f-065c-4b63-8f9c-1b90a26fba71 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:04:52.338322 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-8200c33f-065c-4b63-8f9c-1b90a26fba71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:04:52.361643 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:04:52.403691 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 29/59] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:04:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:04:52.427721 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da42ceba-5c1c-4602-82c8-ea6ffbdadbe1 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:52.520289 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:04:52.528043 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da42ceba-5c1c-4602-82c8-ea6ffbdadbe1 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:52.560944 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e8253d-29fe-4af9-9d16-700a733e8358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:52.561743 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e8253d-29fe-4af9-9d16-700a733e8358 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:52.562031 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e8253d-29fe-4af9-9d16-700a733e8358 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:52.562310 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e8253d-29fe-4af9-9d16-700a733e8358 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:52.563545 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e8253d-29fe-4af9-9d16-700a733e8358 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:04:52.564986 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e8253d-29fe-4af9-9d16-700a733e8358 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:04:52.566605 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 31/60] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:04:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:04:52.572127 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8200c33f-065c-4b63-8f9c-1b90a26fba71 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:52.669537 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8200c33f-065c-4b63-8f9c-1b90a26fba71 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:52.702942 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063bd4c3-b8b5-4ade-9b5e-76f03393cc75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:52.703616 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063bd4c3-b8b5-4ade-9b5e-76f03393cc75 None admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:52.703905 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063bd4c3-b8b5-4ade-9b5e-76f03393cc75 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:52.704185 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063bd4c3-b8b5-4ade-9b5e-76f03393cc75 None admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:52.704937 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063bd4c3-b8b5-4ade-9b5e-76f03393cc75 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:04:52.706400 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063bd4c3-b8b5-4ade-9b5e-76f03393cc75 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:04:52.804117 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 30/61] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Jun 21 18:04:52 2022] POST /identity/v3/users => generated 275 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:04:56.025104 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cbe59c-eee6-44b2-b23c-6bdb8977806d None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:56.026222 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cbe59c-eee6-44b2-b23c-6bdb8977806d None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:56.026687 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cbe59c-eee6-44b2-b23c-6bdb8977806d None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:56.028261 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 32/62] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:04:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:04:56.033845 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e78f20e-5803-4ecd-9f72-6fc6cb506edf None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:56.034814 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e78f20e-5803-4ecd-9f72-6fc6cb506edf None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:56.035278 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e78f20e-5803-4ecd-9f72-6fc6cb506edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:56.061487 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-1e78f20e-5803-4ecd-9f72-6fc6cb506edf None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:04:56.062993 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-1e78f20e-5803-4ecd-9f72-6fc6cb506edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:04:56.082137 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:04:56.157509 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 31/63] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:04:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:04:56.196584 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cbe59c-eee6-44b2-b23c-6bdb8977806d None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:56.345085 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cbe59c-eee6-44b2-b23c-6bdb8977806d None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:56.397420 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c1c3cf-ea28-4fe3-94e7-c5e8d0fcd6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:56.398475 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c1c3cf-ea28-4fe3-94e7-c5e8d0fcd6ba None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:56.398929 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c1c3cf-ea28-4fe3-94e7-c5e8d0fcd6ba None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:56.399348 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c1c3cf-ea28-4fe3-94e7-c5e8d0fcd6ba None admin] PATH_INFO: `/v3/roles/member` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:56.417921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c1c3cf-ea28-4fe3-94e7-c5e8d0fcd6ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:04:56.426032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-20c1c3cf-ea28-4fe3-94e7-c5e8d0fcd6ba None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:04:56.426032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:04:56.426032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:04:56.426032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:04:56.426032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:04:56.426032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:04:56.426032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:04:56.426032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:04:56.426032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:04:56.426032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:04:56.426032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:04:56.426032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:04:56.426032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:04:56.426032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:04:56.426032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:04:56.426032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:04:56.427176 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:04:56.427176 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:04:56.427176 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:04:56.427176 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:04:56.427176 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:04:56.427176 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:04:56.427176 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:04:56.427176 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:04:56.427176 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:04:56.427176 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:04:56.427176 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:04:56.427176 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:04:56.427176 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:04:56.427176 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:04:56.427176 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:04:56.427176 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:04:56.428172 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:04:56.428172 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:04:56.428172 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:04:56.428172 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:04:56.428172 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:04:56.428172 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:04:56.428172 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:04:56.428172 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:04:56.428172 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:04:56.428172 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:04:56.428172 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:04:56.428172 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:04:56.430293 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 33/64] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:04:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:04:56.437183 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e78f20e-5803-4ecd-9f72-6fc6cb506edf None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:56.493354 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caafaefa-f18b-402b-b9d3-e51c70cc483f None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:56.494009 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caafaefa-f18b-402b-b9d3-e51c70cc483f None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:56.494657 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caafaefa-f18b-402b-b9d3-e51c70cc483f None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:56.497010 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 34/65] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:04:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:04:56.502740 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba4ad47-33e2-4748-a201-f7754bfaca13 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:56.503324 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba4ad47-33e2-4748-a201-f7754bfaca13 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:56.503816 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba4ad47-33e2-4748-a201-f7754bfaca13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:56.537446 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-5ba4ad47-33e2-4748-a201-f7754bfaca13 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:04:56.539350 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-5ba4ad47-33e2-4748-a201-f7754bfaca13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:04:56.586992 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:04:56.589088 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e78f20e-5803-4ecd-9f72-6fc6cb506edf None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:56.640331 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048d2ff4-160b-4e61-b7da-b5fa3046c765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:56.641064 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d2ff4-160b-4e61-b7da-b5fa3046c765 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:56.641351 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d2ff4-160b-4e61-b7da-b5fa3046c765 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:56.641632 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d2ff4-160b-4e61-b7da-b5fa3046c765 None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:56.642732 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048d2ff4-160b-4e61-b7da-b5fa3046c765 None admin] RBAC: Adding query filter params (name=member) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:04:56.643240 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048d2ff4-160b-4e61-b7da-b5fa3046c765 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:04:56.645099 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048d2ff4-160b-4e61-b7da-b5fa3046c765 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:04:56.650399 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 35/66] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:04:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:04:56.655037 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 32/67] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jun 21 18:04:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:04:56.656542 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47008182-4344-4322-a76c-1c0f31e7a556 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:56.659772 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47008182-4344-4322-a76c-1c0f31e7a556 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:56.659772 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47008182-4344-4322-a76c-1c0f31e7a556 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:56.659772 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 33/68] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:04:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:04:56.660802 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba4ad47-33e2-4748-a201-f7754bfaca13 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:56.670927 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aaf806-6e19-4fd5-bdb3-52194031d7bb None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:56.671208 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aaf806-6e19-4fd5-bdb3-52194031d7bb None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:56.671472 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aaf806-6e19-4fd5-bdb3-52194031d7bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:56.691467 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-99aaf806-6e19-4fd5-bdb3-52194031d7bb None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:04:56.692462 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-99aaf806-6e19-4fd5-bdb3-52194031d7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:04:56.694846 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:04:56.786149 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 34/69] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:04:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:04:56.786149 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aaf806-6e19-4fd5-bdb3-52194031d7bb None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:56.801006 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba4ad47-33e2-4748-a201-f7754bfaca13 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:56.848072 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970cba55-dae3-47d9-a123-559f849a006a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:56.849050 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970cba55-dae3-47d9-a123-559f849a006a None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:56.849397 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970cba55-dae3-47d9-a123-559f849a006a None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:56.849653 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970cba55-dae3-47d9-a123-559f849a006a None admin] PATH_INFO: `/v3/users/623a005b923a4f81bdcab9453639598a` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:56.864522 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970cba55-dae3-47d9-a123-559f849a006a None admin] RBAC: Authorizing `identity:get_user(user_id=623a005b923a4f81bdcab9453639598a)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:04:56.866036 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970cba55-dae3-47d9-a123-559f849a006a None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:04:56.867784 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 36/70] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jun 21 18:04:56 2022] GET /identity/v3/users/623a005b923a4f81bdcab9453639598a => generated 275 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:04:56.869097 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1a3469-6f89-4018-8926-61dadd3cf52d None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:56.869387 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1a3469-6f89-4018-8926-61dadd3cf52d None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:56.870208 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1a3469-6f89-4018-8926-61dadd3cf52d None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:56.875453 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 37/71] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:04:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:04:56.875453 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1a3469-6f89-4018-8926-61dadd3cf52d None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:56.887055 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:04:56.909862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aaf806-6e19-4fd5-bdb3-52194031d7bb None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:56.944231 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daefcfd-1959-4e14-be2d-4ef68ebff912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:56.944983 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daefcfd-1959-4e14-be2d-4ef68ebff912 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:56.945281 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daefcfd-1959-4e14-be2d-4ef68ebff912 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:56.945567 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daefcfd-1959-4e14-be2d-4ef68ebff912 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:56.953742 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:04:56.957879 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daefcfd-1959-4e14-be2d-4ef68ebff912 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:04:56.966002 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-1daefcfd-1959-4e14-be2d-4ef68ebff912 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:04:56.966002 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:04:56.966002 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:04:56.966002 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:04:56.966002 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:04:56.966002 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:04:56.966002 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:04:56.966002 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:04:56.966002 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:04:56.966002 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:04:56.966002 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:04:56.966002 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:04:56.966002 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:04:56.966002 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:04:56.966002 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:04:56.966002 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:04:56.966998 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:04:56.966998 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:04:56.966998 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:04:56.966998 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:04:56.966998 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:04:56.966998 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:04:56.966998 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:04:56.966998 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:04:56.966998 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:04:56.966998 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:04:56.966998 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:04:56.966998 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:04:56.966998 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:04:56.966998 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:04:56.966998 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:04:56.966998 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:04:56.967773 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:04:56.967773 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:04:56.967773 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:04:56.967773 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:04:56.967773 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:04:56.967773 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:04:56.967773 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:04:56.967773 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:04:56.967773 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:04:56.967773 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:04:56.967773 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:04:56.967773 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:04:56.967773 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 35/72] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jun 21 18:04:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:04:56.968839 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe57332-5225-4a73-9f78-6a254a8ee342 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:56.969159 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe57332-5225-4a73-9f78-6a254a8ee342 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:56.969385 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe57332-5225-4a73-9f78-6a254a8ee342 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:56.970721 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 36/73] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:04:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:04:56.971441 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe57332-5225-4a73-9f78-6a254a8ee342 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:56.986981 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1a3469-6f89-4018-8926-61dadd3cf52d None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:57.021922 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a410ecf-cba4-498b-9bcb-d595b35a196c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:57.022736 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a410ecf-cba4-498b-9bcb-d595b35a196c None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:57.023024 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a410ecf-cba4-498b-9bcb-d595b35a196c None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:57.023307 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a410ecf-cba4-498b-9bcb-d595b35a196c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:57.034266 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a410ecf-cba4-498b-9bcb-d595b35a196c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:04:57.042106 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-7a410ecf-cba4-498b-9bcb-d595b35a196c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:04:57.042106 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:04:57.042106 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:04:57.042106 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:04:57.042106 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:04:57.042106 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:04:57.042106 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:04:57.042106 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:04:57.042106 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:04:57.042106 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:04:57.042106 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:04:57.042106 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:04:57.042106 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:04:57.042106 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:04:57.042106 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:04:57.042106 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:04:57.043304 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:04:57.043304 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:04:57.043304 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:04:57.043304 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:04:57.043304 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:04:57.043304 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:04:57.043304 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:04:57.043304 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:04:57.043304 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:04:57.043304 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:04:57.043304 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:04:57.043304 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:04:57.043304 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:04:57.043304 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:04:57.043304 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:04:57.043304 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:04:57.044125 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:04:57.044125 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:04:57.044125 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:04:57.044125 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:04:57.044125 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:04:57.044125 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:04:57.044125 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:04:57.044125 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:04:57.044125 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:04:57.044125 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:04:57.044125 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:04:57.044125 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:04:57.044125 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 38/74] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:04:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:04:57.044843 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3247e4e-00e6-4e9b-a06f-66729ca98363 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:57.044989 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3247e4e-00e6-4e9b-a06f-66729ca98363 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:57.045276 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3247e4e-00e6-4e9b-a06f-66729ca98363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:57.064056 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-e3247e4e-00e6-4e9b-a06f-66729ca98363 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:04:57.065285 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-e3247e4e-00e6-4e9b-a06f-66729ca98363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:04:57.081021 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe57332-5225-4a73-9f78-6a254a8ee342 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:57.127678 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59930b7a-aab2-4718-bbfa-e14a49782886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:57.128358 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59930b7a-aab2-4718-bbfa-e14a49782886 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:57.128660 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59930b7a-aab2-4718-bbfa-e14a49782886 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:57.128943 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59930b7a-aab2-4718-bbfa-e14a49782886 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:57.130130 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:04:57.134652 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 39/75] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:04:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:04:57.136112 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3247e4e-00e6-4e9b-a06f-66729ca98363 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:57.200173 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:04:57.203634 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59930b7a-aab2-4718-bbfa-e14a49782886 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:04:57.205156 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59930b7a-aab2-4718-bbfa-e14a49782886 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:04:57.238447 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3247e4e-00e6-4e9b-a06f-66729ca98363 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:57.272099 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaadd9b1-ce7f-414d-9e12-e2eee445a80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:57.272804 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaadd9b1-ce7f-414d-9e12-e2eee445a80e None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:57.273094 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaadd9b1-ce7f-414d-9e12-e2eee445a80e None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:57.273371 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaadd9b1-ce7f-414d-9e12-e2eee445a80e None admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:57.274116 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaadd9b1-ce7f-414d-9e12-e2eee445a80e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:04:57.274424 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaadd9b1-ce7f-414d-9e12-e2eee445a80e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:04:57.275876 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaadd9b1-ce7f-414d-9e12-e2eee445a80e None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:04:57.284924 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 40/76] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:04:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:04:57.286281 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e590ec9-c1d4-4cd5-a679-0b5668c2409b None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:57.286576 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e590ec9-c1d4-4cd5-a679-0b5668c2409b None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:57.286855 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e590ec9-c1d4-4cd5-a679-0b5668c2409b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:57.302971 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-3e590ec9-c1d4-4cd5-a679-0b5668c2409b None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:04:57.304162 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-3e590ec9-c1d4-4cd5-a679-0b5668c2409b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:04:57.304682 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 37/77] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:04:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:04:57.305546 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59930b7a-aab2-4718-bbfa-e14a49782886 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:57.367803 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:04:57.378518 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 41/78] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:04:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:04:57.379360 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e590ec9-c1d4-4cd5-a679-0b5668c2409b None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:57.409620 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:04:57.413721 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59930b7a-aab2-4718-bbfa-e14a49782886 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:57.462184 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e9ea57-bd88-4dc3-b23b-3b024666a1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:57.462783 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e9ea57-bd88-4dc3-b23b-3b024666a1f1 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:57.463073 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e9ea57-bd88-4dc3-b23b-3b024666a1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:57.463414 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e9ea57-bd88-4dc3-b23b-3b024666a1f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:57.464346 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e9ea57-bd88-4dc3-b23b-3b024666a1f1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:04:57.464695 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e9ea57-bd88-4dc3-b23b-3b024666a1f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:04:57.466120 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e9ea57-bd88-4dc3-b23b-3b024666a1f1 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:04:57.474792 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 38/79] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jun 21 18:04:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:04:57.477305 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e9ea57-bd88-4dc3-b23b-3b024666a1f1 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:57.491051 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e590ec9-c1d4-4cd5-a679-0b5668c2409b None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:57.524701 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fa2533-df6b-47ed-9211-9eef33e6d886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:57.525541 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fa2533-df6b-47ed-9211-9eef33e6d886 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:57.526339 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fa2533-df6b-47ed-9211-9eef33e6d886 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:57.526622 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fa2533-df6b-47ed-9211-9eef33e6d886 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:57.536724 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fa2533-df6b-47ed-9211-9eef33e6d886 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:04:57.545487 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-d9fa2533-df6b-47ed-9211-9eef33e6d886 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:04:57.545487 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:04:57.545487 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:04:57.545487 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:04:57.545487 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:04:57.545487 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:04:57.545487 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:04:57.545487 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:04:57.545487 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:04:57.545487 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:04:57.545487 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:04:57.545487 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:04:57.545487 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:04:57.545487 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:04:57.545487 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:04:57.545487 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:04:57.546501 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:04:57.546501 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:04:57.546501 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:04:57.546501 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:04:57.546501 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:04:57.546501 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:04:57.546501 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:04:57.546501 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:04:57.546501 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:04:57.546501 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:04:57.546501 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:04:57.546501 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:04:57.546501 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:04:57.546501 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:04:57.546501 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:04:57.546501 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:04:57.547320 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:04:57.547320 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:04:57.547320 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:04:57.547320 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:04:57.547320 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:04:57.547320 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:04:57.547320 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:04:57.547320 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:04:57.547320 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:04:57.547320 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:04:57.547320 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:04:57.547320 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:04:57.547320 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 42/80] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jun 21 18:04:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:04:57.547976 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fa2533-df6b-47ed-9211-9eef33e6d886 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:57.583760 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e9ea57-bd88-4dc3-b23b-3b024666a1f1 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:57.600921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:04:57.618605 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093ffd67-a6f6-4b5b-98bf-e4fc3fa6292f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:57.619573 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093ffd67-a6f6-4b5b-98bf-e4fc3fa6292f None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:57.619995 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093ffd67-a6f6-4b5b-98bf-e4fc3fa6292f None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:57.620256 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093ffd67-a6f6-4b5b-98bf-e4fc3fa6292f None admin] PATH_INFO: `/v3/users/0c89ac265bd34c75a692d91d672fad59` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:57.621536 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093ffd67-a6f6-4b5b-98bf-e4fc3fa6292f None admin] RBAC: Authorizing `identity:get_user(user_id=0c89ac265bd34c75a692d91d672fad59)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:04:57.622993 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093ffd67-a6f6-4b5b-98bf-e4fc3fa6292f None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:04:57.624691 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 39/81] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jun 21 18:04:57 2022] GET /identity/v3/users/0c89ac265bd34c75a692d91d672fad59 => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:04:57.625466 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093ffd67-a6f6-4b5b-98bf-e4fc3fa6292f None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:57.650198 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fa2533-df6b-47ed-9211-9eef33e6d886 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:57.657037 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:04:57.685548 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e08bebe-93e1-4bf2-a5f1-486f421cf759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:57.686416 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e08bebe-93e1-4bf2-a5f1-486f421cf759 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:57.686707 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e08bebe-93e1-4bf2-a5f1-486f421cf759 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:57.686990 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e08bebe-93e1-4bf2-a5f1-486f421cf759 None admin] PATH_INFO: `/v3/projects/21cd4117dcd346b8bcb3e353ff8d228a` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:57.688199 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e08bebe-93e1-4bf2-a5f1-486f421cf759 None admin] RBAC: Authorizing `identity:get_project(project_id=21cd4117dcd346b8bcb3e353ff8d228a)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:04:57.689713 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e08bebe-93e1-4bf2-a5f1-486f421cf759 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:04:57.691434 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 43/82] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jun 21 18:04:57 2022] GET /identity/v3/projects/21cd4117dcd346b8bcb3e353ff8d228a => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:04:57.692212 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e08bebe-93e1-4bf2-a5f1-486f421cf759 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:57.730159 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093ffd67-a6f6-4b5b-98bf-e4fc3fa6292f None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:57.763485 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e19c7e-3693-4028-bb4b-4fd1df38c3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:57.764292 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e19c7e-3693-4028-bb4b-4fd1df38c3c0 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:57.764608 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e19c7e-3693-4028-bb4b-4fd1df38c3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:57.764890 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e19c7e-3693-4028-bb4b-4fd1df38c3c0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:57.766103 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e19c7e-3693-4028-bb4b-4fd1df38c3c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:04:57.767555 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e19c7e-3693-4028-bb4b-4fd1df38c3c0 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:04:57.769227 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 40/83] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:04:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:04:57.770012 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e19c7e-3693-4028-bb4b-4fd1df38c3c0 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:57.796706 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e08bebe-93e1-4bf2-a5f1-486f421cf759 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:57.817980 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:04:57.832097 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166d82bf-4efc-443a-890b-b1313cd9c4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:57.833007 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166d82bf-4efc-443a-890b-b1313cd9c4d7 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:57.833295 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166d82bf-4efc-443a-890b-b1313cd9c4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:57.833576 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166d82bf-4efc-443a-890b-b1313cd9c4d7 None admin] PATH_INFO: `/v3/users/623a005b923a4f81bdcab9453639598a` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:57.836263 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166d82bf-4efc-443a-890b-b1313cd9c4d7 None admin] RBAC: Authorizing `identity:get_user(user_id=623a005b923a4f81bdcab9453639598a)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:04:57.836263 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166d82bf-4efc-443a-890b-b1313cd9c4d7 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:04:57.838042 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 44/84] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jun 21 18:04:57 2022] GET /identity/v3/users/623a005b923a4f81bdcab9453639598a => generated 275 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:04:57.838841 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166d82bf-4efc-443a-890b-b1313cd9c4d7 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:57.873301 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:04:57.879057 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e19c7e-3693-4028-bb4b-4fd1df38c3c0 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:57.912334 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1a0a52-e4e5-4a0c-8238-700bbda79b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:57.913052 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a0a52-e4e5-4a0c-8238-700bbda79b4d None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:57.913349 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a0a52-e4e5-4a0c-8238-700bbda79b4d None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:57.913636 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a0a52-e4e5-4a0c-8238-700bbda79b4d None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:57.914398 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1a0a52-e4e5-4a0c-8238-700bbda79b4d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:04:57.914713 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1a0a52-e4e5-4a0c-8238-700bbda79b4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:04:57.916119 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1a0a52-e4e5-4a0c-8238-700bbda79b4d None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:04:57.925276 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 41/85] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jun 21 18:04:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:04:57.926156 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1a0a52-e4e5-4a0c-8238-700bbda79b4d None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:57.947966 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166d82bf-4efc-443a-890b-b1313cd9c4d7 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:57.982225 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bac34e7-26bb-46b8-96eb-90919ac62dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:57.982792 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac34e7-26bb-46b8-96eb-90919ac62dc6 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:57.983083 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac34e7-26bb-46b8-96eb-90919ac62dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:57.983364 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac34e7-26bb-46b8-96eb-90919ac62dc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:58.031933 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1a0a52-e4e5-4a0c-8238-700bbda79b4d None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:58.038842 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:04:58.052460 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bac34e7-26bb-46b8-96eb-90919ac62dc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:04:58.053934 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bac34e7-26bb-46b8-96eb-90919ac62dc6 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:04:58.067443 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdb85dd-35dd-4ade-8de2-3d3d4aa4d1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:58.068112 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb85dd-35dd-4ade-8de2-3d3d4aa4d1c5 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:58.068426 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb85dd-35dd-4ade-8de2-3d3d4aa4d1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:58.068719 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb85dd-35dd-4ade-8de2-3d3d4aa4d1c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:58.069565 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdb85dd-35dd-4ade-8de2-3d3d4aa4d1c5 None admin] RBAC: Adding query filter params (role.id=c467f10d3dba4f2dbfd42068f42f74b6, scope.project.id=21cd4117dcd346b8bcb3e353ff8d228a, user.id=623a005b923a4f81bdcab9453639598a) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:04:58.069883 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdb85dd-35dd-4ade-8de2-3d3d4aa4d1c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:04:58.071432 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdb85dd-35dd-4ade-8de2-3d3d4aa4d1c5 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:04:58.079873 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 42/86] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jun 21 18:04:57 2022] GET /identity/v3/role_assignments?user.id=623a005b923a4f81bdcab9453639598a&scope.project.id=21cd4117dcd346b8bcb3e353ff8d228a&role.id=c467f10d3dba4f2dbfd42068f42f74b6 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:04:58.080702 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdb85dd-35dd-4ade-8de2-3d3d4aa4d1c5 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:58.118524 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:04:58.165526 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 45/87] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:04:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:04:58.166499 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bac34e7-26bb-46b8-96eb-90919ac62dc6 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:58.220538 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdb85dd-35dd-4ade-8de2-3d3d4aa4d1c5 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:58.258508 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d69b1ac-d4b3-43da-a1ee-eaf6da106e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:58.259119 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d69b1ac-d4b3-43da-a1ee-eaf6da106e2e None admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:58.259413 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d69b1ac-d4b3-43da-a1ee-eaf6da106e2e None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:58.259698 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d69b1ac-d4b3-43da-a1ee-eaf6da106e2e None admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:58.261023 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d69b1ac-d4b3-43da-a1ee-eaf6da106e2e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:04:58.262491 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d69b1ac-d4b3-43da-a1ee-eaf6da106e2e None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:04:58.279832 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:04:58.292755 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bac34e7-26bb-46b8-96eb-90919ac62dc6 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:58.329130 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7765b09d-9c5a-4272-a823-d9caa9c56530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:58.329823 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7765b09d-9c5a-4272-a823-d9caa9c56530 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:58.330122 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7765b09d-9c5a-4272-a823-d9caa9c56530 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:58.330405 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7765b09d-9c5a-4272-a823-d9caa9c56530 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:58.341967 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 43/88] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Jun 21 18:04:58 2022] POST /identity/v3/users => generated 289 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:04:58.342852 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d69b1ac-d4b3-43da-a1ee-eaf6da106e2e None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:58.454552 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:04:58.467572 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7765b09d-9c5a-4272-a823-d9caa9c56530 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:04:58.470608 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7765b09d-9c5a-4272-a823-d9caa9c56530 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:04:58.488831 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d69b1ac-d4b3-43da-a1ee-eaf6da106e2e None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:58.547297 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857406b7-0dc8-47ed-a3bd-5b8a04a5a40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:58.548669 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857406b7-0dc8-47ed-a3bd-5b8a04a5a40a None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:58.549619 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857406b7-0dc8-47ed-a3bd-5b8a04a5a40a None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:58.550242 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857406b7-0dc8-47ed-a3bd-5b8a04a5a40a None admin] PATH_INFO: `/v3/users/623a005b923a4f81bdcab9453639598a` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:58.552327 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857406b7-0dc8-47ed-a3bd-5b8a04a5a40a None admin] RBAC: Authorizing `identity:get_user(user_id=623a005b923a4f81bdcab9453639598a)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:04:58.555668 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857406b7-0dc8-47ed-a3bd-5b8a04a5a40a None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:04:58.556517 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 44/89] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jun 21 18:04:58 2022] GET /identity/v3/users/623a005b923a4f81bdcab9453639598a => generated 275 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:04:58.557813 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857406b7-0dc8-47ed-a3bd-5b8a04a5a40a None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:58.591488 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:04:58.611759 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 46/90] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:04:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:04:58.613090 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7765b09d-9c5a-4272-a823-d9caa9c56530 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:58.689410 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857406b7-0dc8-47ed-a3bd-5b8a04a5a40a None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:58.731007 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886fc7a-dc1b-4de5-a10d-8327be7788c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:58.732277 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886fc7a-dc1b-4de5-a10d-8327be7788c3 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:58.732277 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886fc7a-dc1b-4de5-a10d-8327be7788c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:58.732595 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886fc7a-dc1b-4de5-a10d-8327be7788c3 None admin] PATH_INFO: `/v3/projects/21cd4117dcd346b8bcb3e353ff8d228a` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:58.733884 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e886fc7a-dc1b-4de5-a10d-8327be7788c3 None admin] RBAC: Authorizing `identity:get_project(project_id=21cd4117dcd346b8bcb3e353ff8d228a)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:04:58.735462 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e886fc7a-dc1b-4de5-a10d-8327be7788c3 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:04:58.742109 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7765b09d-9c5a-4272-a823-d9caa9c56530 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:58.742396 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 45/91] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jun 21 18:04:58 2022] GET /identity/v3/projects/21cd4117dcd346b8bcb3e353ff8d228a => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:04:58.743364 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886fc7a-dc1b-4de5-a10d-8327be7788c3 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:58.761754 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:04:58.784702 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea33cad6-f1c5-446b-baed-d934879818a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:58.785725 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea33cad6-f1c5-446b-baed-d934879818a9 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:58.786258 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea33cad6-f1c5-446b-baed-d934879818a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:58.786730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea33cad6-f1c5-446b-baed-d934879818a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:58.852456 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:04:58.860833 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886fc7a-dc1b-4de5-a10d-8327be7788c3 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:58.867039 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea33cad6-f1c5-446b-baed-d934879818a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:04:58.868858 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea33cad6-f1c5-446b-baed-d934879818a9 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:04:58.898358 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8449a484-c1ef-438e-88d7-74aa0777f6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:58.899250 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8449a484-c1ef-438e-88d7-74aa0777f6bc None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:58.899648 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8449a484-c1ef-438e-88d7-74aa0777f6bc None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:58.899942 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8449a484-c1ef-438e-88d7-74aa0777f6bc None admin] PATH_INFO: `/v3/projects/243a6a54d0b6415e948330ba3cd64d7e` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:58.902085 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8449a484-c1ef-438e-88d7-74aa0777f6bc None admin] RBAC: Authorizing `identity:get_project(project_id=243a6a54d0b6415e948330ba3cd64d7e)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:04:58.903670 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8449a484-c1ef-438e-88d7-74aa0777f6bc None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:04:58.905923 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 46/92] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jun 21 18:04:58 2022] GET /identity/v3/projects/243a6a54d0b6415e948330ba3cd64d7e => generated 312 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:04:58.906773 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8449a484-c1ef-438e-88d7-74aa0777f6bc None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:58.987119 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 47/93] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:04:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:04:58.988198 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea33cad6-f1c5-446b-baed-d934879818a9 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:59.000253 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:04:59.041379 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8449a484-c1ef-438e-88d7-74aa0777f6bc None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:59.093636 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:04:59.101032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea33cad6-f1c5-446b-baed-d934879818a9 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:59.136344 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548892a9-f5a6-4736-83c7-12931ed0cb6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:59.137498 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548892a9-f5a6-4736-83c7-12931ed0cb6c None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:59.137797 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548892a9-f5a6-4736-83c7-12931ed0cb6c None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:59.138126 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548892a9-f5a6-4736-83c7-12931ed0cb6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:59.139136 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548892a9-f5a6-4736-83c7-12931ed0cb6c None admin] RBAC: Adding query filter params (role.id=745ed9065a8143e384ebbd694b29635c, scope.project.id=21cd4117dcd346b8bcb3e353ff8d228a, user.id=0c89ac265bd34c75a692d91d672fad59) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:04:59.139482 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548892a9-f5a6-4736-83c7-12931ed0cb6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:04:59.141211 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548892a9-f5a6-4736-83c7-12931ed0cb6c None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:04:59.141715 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55af2462-f207-4d9a-b933-a3b783674236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:59.142528 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55af2462-f207-4d9a-b933-a3b783674236 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:59.142816 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55af2462-f207-4d9a-b933-a3b783674236 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:59.143280 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55af2462-f207-4d9a-b933-a3b783674236 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:59.144126 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55af2462-f207-4d9a-b933-a3b783674236 None admin] RBAC: Adding query filter params (role.id=c467f10d3dba4f2dbfd42068f42f74b6, scope.project.id=243a6a54d0b6415e948330ba3cd64d7e, user.id=623a005b923a4f81bdcab9453639598a) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:04:59.144919 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55af2462-f207-4d9a-b933-a3b783674236 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:04:59.146356 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55af2462-f207-4d9a-b933-a3b783674236 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:04:59.153131 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 48/94] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jun 21 18:04:58 2022] GET /identity/v3/role_assignments?user.id=623a005b923a4f81bdcab9453639598a&scope.project.id=243a6a54d0b6415e948330ba3cd64d7e&role.id=c467f10d3dba4f2dbfd42068f42f74b6 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:04:59.153968 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55af2462-f207-4d9a-b933-a3b783674236 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:59.155160 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 47/95] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jun 21 18:04:58 2022] GET /identity/v3/role_assignments?user.id=0c89ac265bd34c75a692d91d672fad59&scope.project.id=21cd4117dcd346b8bcb3e353ff8d228a&role.id=745ed9065a8143e384ebbd694b29635c => generated 259 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:04:59.291061 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55af2462-f207-4d9a-b933-a3b783674236 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:59.332303 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e5e881-7d89-467a-8b82-9971b361dd32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:59.333210 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e5e881-7d89-467a-8b82-9971b361dd32 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:59.333496 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e5e881-7d89-467a-8b82-9971b361dd32 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:59.333774 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e5e881-7d89-467a-8b82-9971b361dd32 None admin] PATH_INFO: `/v3/projects/21cd4117dcd346b8bcb3e353ff8d228a` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:59.335045 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e5e881-7d89-467a-8b82-9971b361dd32 None admin] RBAC: Authorizing `identity:get_project(project_id=21cd4117dcd346b8bcb3e353ff8d228a)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:04:59.336585 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e5e881-7d89-467a-8b82-9971b361dd32 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:04:59.338667 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 49/96] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jun 21 18:04:59 2022] GET /identity/v3/projects/21cd4117dcd346b8bcb3e353ff8d228a => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:04:59.344600 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548892a9-f5a6-4736-83c7-12931ed0cb6c None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:04:59.462270 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548892a9-f5a6-4736-83c7-12931ed0cb6c None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:04:59.501682 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a233e737-8766-45ae-a652-1d1a6654a9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:04:59.502363 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a233e737-8766-45ae-a652-1d1a6654a9a1 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:04:59.502651 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a233e737-8766-45ae-a652-1d1a6654a9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:04:59.502932 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a233e737-8766-45ae-a652-1d1a6654a9a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:04:59.503787 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a233e737-8766-45ae-a652-1d1a6654a9a1 None admin] RBAC: Adding query filter params (role.id=482dcc9ba0fd400cad2d193e3e895d4c, scope.project.id=21cd4117dcd346b8bcb3e353ff8d228a, user.id=623a005b923a4f81bdcab9453639598a) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:04:59.504097 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a233e737-8766-45ae-a652-1d1a6654a9a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:04:59.505641 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a233e737-8766-45ae-a652-1d1a6654a9a1 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:04:59.512523 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 48/97] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jun 21 18:04:59 2022] GET /identity/v3/role_assignments?user.id=623a005b923a4f81bdcab9453639598a&scope.project.id=21cd4117dcd346b8bcb3e353ff8d228a&role.id=482dcc9ba0fd400cad2d193e3e895d4c => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:00.478201 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb185ae1-5abc-4ab8-96c4-79a52e3c4f5d None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:00.478721 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb185ae1-5abc-4ab8-96c4-79a52e3c4f5d None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:00.478796 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb185ae1-5abc-4ab8-96c4-79a52e3c4f5d None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:00.480783 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 50/98] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:05:00.486392 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a86ea-f507-4f1a-aaa4-1593d3b4f4d7 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:00.486875 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a86ea-f507-4f1a-aaa4-1593d3b4f4d7 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:00.486978 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a86ea-f507-4f1a-aaa4-1593d3b4f4d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:00.505110 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-7d9a86ea-f507-4f1a-aaa4-1593d3b4f4d7 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:00.506307 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-7d9a86ea-f507-4f1a-aaa4-1593d3b4f4d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:00.509292 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:00.581031 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 49/99] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:05:00.604764 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb185ae1-5abc-4ab8-96c4-79a52e3c4f5d None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:00.627843 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:00.712720 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb185ae1-5abc-4ab8-96c4-79a52e3c4f5d None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:00.750962 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a831c3db-31f2-4485-a7f5-fbd6a8bdce3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:00.751723 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a831c3db-31f2-4485-a7f5-fbd6a8bdce3c None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:00.752008 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a831c3db-31f2-4485-a7f5-fbd6a8bdce3c None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:00.752287 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a831c3db-31f2-4485-a7f5-fbd6a8bdce3c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:00.765221 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a831c3db-31f2-4485-a7f5-fbd6a8bdce3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:05:00.772942 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-a831c3db-31f2-4485-a7f5-fbd6a8bdce3c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:05:00.772942 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:00.772942 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:00.772942 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:00.772942 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:00.772942 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:00.772942 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:00.772942 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:00.772942 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:00.772942 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:00.772942 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:00.772942 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:00.772942 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:05:00.772942 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:05:00.772942 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:05:00.772942 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:05:00.775366 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:05:00.775366 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:05:00.775366 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:05:00.775366 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:05:00.775366 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:00.775366 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:00.775366 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:00.775366 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:00.775366 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:00.775366 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:00.775366 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:00.775366 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:00.775366 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:00.775366 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:00.775366 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:00.775366 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:00.777828 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:00.777828 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:00.777828 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:00.777828 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:00.777828 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:05:00.777828 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:05:00.777828 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:05:00.777828 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:05:00.777828 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:05:00.777828 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:05:00.777828 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:05:00.777828 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:05:00.777828 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 51/100] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:05:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:00.784467 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9a86ea-f507-4f1a-aaa4-1593d3b4f4d7 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:00.883032 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9a86ea-f507-4f1a-aaa4-1593d3b4f4d7 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:00.922227 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f01633-0ee7-4601-8a24-2574d7b89903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:00.922932 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f01633-0ee7-4601-8a24-2574d7b89903 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:00.923209 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f01633-0ee7-4601-8a24-2574d7b89903 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:00.923494 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f01633-0ee7-4601-8a24-2574d7b89903 None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:00.924269 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f01633-0ee7-4601-8a24-2574d7b89903 None admin] RBAC: Adding query filter params (name=member) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:00.924619 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f01633-0ee7-4601-8a24-2574d7b89903 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:00.926267 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f01633-0ee7-4601-8a24-2574d7b89903 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:00.936558 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 50/101] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jun 21 18:05:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:00.942222 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a831c3db-31f2-4485-a7f5-fbd6a8bdce3c None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:01.033826 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:01.054517 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a831c3db-31f2-4485-a7f5-fbd6a8bdce3c None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:01.093014 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f4eab3-679b-40ad-bcd5-d11194c81736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:01.093895 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f4eab3-679b-40ad-bcd5-d11194c81736 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:01.094179 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f4eab3-679b-40ad-bcd5-d11194c81736 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:01.094454 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f4eab3-679b-40ad-bcd5-d11194c81736 None admin] PATH_INFO: `/v3/users/623a005b923a4f81bdcab9453639598a` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:01.095664 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f4eab3-679b-40ad-bcd5-d11194c81736 None admin] RBAC: Authorizing `identity:get_user(user_id=623a005b923a4f81bdcab9453639598a)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:01.097409 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f4eab3-679b-40ad-bcd5-d11194c81736 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:01.099057 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 52/102] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jun 21 18:05:00 2022] GET /identity/v3/users/623a005b923a4f81bdcab9453639598a => generated 275 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:01.104593 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f01633-0ee7-4601-8a24-2574d7b89903 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:01.107521 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:01.208600 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f01633-0ee7-4601-8a24-2574d7b89903 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:01.244241 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ce1133-2686-4b04-95d9-61e4e41c6cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:01.245840 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ce1133-2686-4b04-95d9-61e4e41c6cbf None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:01.246126 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ce1133-2686-4b04-95d9-61e4e41c6cbf None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:01.246406 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ce1133-2686-4b04-95d9-61e4e41c6cbf None admin] PATH_INFO: `/v3/projects/21cd4117dcd346b8bcb3e353ff8d228a` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:01.247657 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ce1133-2686-4b04-95d9-61e4e41c6cbf None admin] RBAC: Authorizing `identity:get_project(project_id=21cd4117dcd346b8bcb3e353ff8d228a)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:01.249141 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ce1133-2686-4b04-95d9-61e4e41c6cbf None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:01.250871 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 51/103] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jun 21 18:05:01 2022] GET /identity/v3/projects/21cd4117dcd346b8bcb3e353ff8d228a => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:01.256740 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f4eab3-679b-40ad-bcd5-d11194c81736 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:01.445470 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f4eab3-679b-40ad-bcd5-d11194c81736 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:01.483060 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a41cd3-6a88-4fc2-9ba0-eb27cd27a634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:01.483966 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a41cd3-6a88-4fc2-9ba0-eb27cd27a634 None admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:01.484250 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a41cd3-6a88-4fc2-9ba0-eb27cd27a634 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:01.484561 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a41cd3-6a88-4fc2-9ba0-eb27cd27a634 None admin] PATH_INFO: `/v3/projects/21cd4117dcd346b8bcb3e353ff8d228a/users/623a005b923a4f81bdcab9453639598a/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:01.486632 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a41cd3-6a88-4fc2-9ba0-eb27cd27a634 None admin] RBAC: Authorizing `identity:create_grant(project_id=21cd4117dcd346b8bcb3e353ff8d228a, user_id=623a005b923a4f81bdcab9453639598a, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:01.488083 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a41cd3-6a88-4fc2-9ba0-eb27cd27a634 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:01.594952 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 53/104] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jun 21 18:05:01 2022] PUT /identity/v3/projects/21cd4117dcd346b8bcb3e353ff8d228a/users/623a005b923a4f81bdcab9453639598a/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:05:01.605254 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57912a6-cd92-400a-8fbe-0a612a6ca7a3 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:01.605995 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57912a6-cd92-400a-8fbe-0a612a6ca7a3 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:01.605995 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57912a6-cd92-400a-8fbe-0a612a6ca7a3 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:01.607420 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 52/105] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:05:01.613003 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ee426-083f-4109-be17-16ff13280e29 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:01.613298 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ee426-083f-4109-be17-16ff13280e29 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:01.613565 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ee426-083f-4109-be17-16ff13280e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:01.641679 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-d90ee426-083f-4109-be17-16ff13280e29 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:01.642916 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-d90ee426-083f-4109-be17-16ff13280e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:01.645383 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:01.731196 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 54/106] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:05:01.755299 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57912a6-cd92-400a-8fbe-0a612a6ca7a3 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:01.848569 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:01.877131 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57912a6-cd92-400a-8fbe-0a612a6ca7a3 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:01.918855 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f802ef17-0b9d-4caf-8707-c0aee941fb08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:01.919718 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f802ef17-0b9d-4caf-8707-c0aee941fb08 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:01.920002 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f802ef17-0b9d-4caf-8707-c0aee941fb08 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:01.920279 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f802ef17-0b9d-4caf-8707-c0aee941fb08 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:01.933739 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f802ef17-0b9d-4caf-8707-c0aee941fb08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:05:01.941709 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-f802ef17-0b9d-4caf-8707-c0aee941fb08 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 18:05:01.941709 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:01.941709 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:01.941709 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:01.941709 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:01.941709 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:01.941709 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:01.941709 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:01.941709 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:01.941709 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:01.941709 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:01.941709 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:01.941709 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:05:01.941709 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:05:01.941709 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:05:01.941709 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:05:01.942710 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:05:01.942710 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:05:01.942710 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:05:01.942710 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:05:01.942710 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:01.942710 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:01.942710 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:01.942710 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:01.942710 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:01.942710 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:01.942710 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:01.942710 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:01.942710 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:01.942710 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:01.942710 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:01.942710 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:01.943546 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:01.943546 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:01.943546 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:01.943546 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:01.943546 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:05:01.943546 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:05:01.943546 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:05:01.943546 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:05:01.943546 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:05:01.943546 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:05:01.943546 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 18:05:01.943546 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:05:01.943546 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 53/107] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:05:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:01.951321 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90ee426-083f-4109-be17-16ff13280e29 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:02.054063 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d32d7-9081-4326-bb19-d5bccc84a9bd None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:02.054678 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d32d7-9081-4326-bb19-d5bccc84a9bd None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:02.055198 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d32d7-9081-4326-bb19-d5bccc84a9bd None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:02.059958 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 54/108] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:05:02.060167 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90ee426-083f-4109-be17-16ff13280e29 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:02.065497 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba7ecf8-d7ae-47b4-8ab1-1e9e4f6ec45b None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:02.066068 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba7ecf8-d7ae-47b4-8ab1-1e9e4f6ec45b None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:02.066551 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba7ecf8-d7ae-47b4-8ab1-1e9e4f6ec45b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:02.094824 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-7ba7ecf8-d7ae-47b4-8ab1-1e9e4f6ec45b None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:02.096770 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-7ba7ecf8-d7ae-47b4-8ab1-1e9e4f6ec45b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:02.098410 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa2314-2f06-4c86-a1c5-a62e8b7e82a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:02.099092 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa2314-2f06-4c86-a1c5-a62e8b7e82a0 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:02.099376 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa2314-2f06-4c86-a1c5-a62e8b7e82a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:02.099655 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa2314-2f06-4c86-a1c5-a62e8b7e82a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:02.103568 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fa2314-2f06-4c86-a1c5-a62e8b7e82a0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:02.103568 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fa2314-2f06-4c86-a1c5-a62e8b7e82a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:02.103568 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fa2314-2f06-4c86-a1c5-a62e8b7e82a0 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:02.108787 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:02.112759 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 55/109] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jun 21 18:05:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:02.122022 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa2314-2f06-4c86-a1c5-a62e8b7e82a0 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:02.245367 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa2314-2f06-4c86-a1c5-a62e8b7e82a0 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:02.250253 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 55/110] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:05:02.281439 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c42d3f-3f7f-49ea-be8d-4fdd4cca33b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:02.282323 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c42d3f-3f7f-49ea-be8d-4fdd4cca33b5 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:02.282604 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c42d3f-3f7f-49ea-be8d-4fdd4cca33b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:02.282875 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c42d3f-3f7f-49ea-be8d-4fdd4cca33b5 None admin] PATH_INFO: `/v3/users/40c2a6f1a2794b24b9a92a919a888051` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:02.297817 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c42d3f-3f7f-49ea-be8d-4fdd4cca33b5 None admin] RBAC: Authorizing `identity:get_user(user_id=40c2a6f1a2794b24b9a92a919a888051)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:02.299366 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c42d3f-3f7f-49ea-be8d-4fdd4cca33b5 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:02.301253 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 56/111] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jun 21 18:05:02 2022] GET /identity/v3/users/40c2a6f1a2794b24b9a92a919a888051 => generated 289 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:02.301587 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba7ecf8-d7ae-47b4-8ab1-1e9e4f6ec45b None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:02.311344 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c42d3f-3f7f-49ea-be8d-4fdd4cca33b5 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:02.402184 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:02.450961 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c42d3f-3f7f-49ea-be8d-4fdd4cca33b5 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:02.461523 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:02.488498 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec81a43d-e1d2-43a4-a4f3-a903c0f30030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:02.489908 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec81a43d-e1d2-43a4-a4f3-a903c0f30030 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:02.490393 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec81a43d-e1d2-43a4-a4f3-a903c0f30030 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:02.490882 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec81a43d-e1d2-43a4-a4f3-a903c0f30030 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:02.508588 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba7ecf8-d7ae-47b4-8ab1-1e9e4f6ec45b None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:02.547416 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4af25e0-6271-421d-8fce-4225d7a3f573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:02.548583 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4af25e0-6271-421d-8fce-4225d7a3f573 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:02.549020 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4af25e0-6271-421d-8fce-4225d7a3f573 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:02.549564 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4af25e0-6271-421d-8fce-4225d7a3f573 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:02.563170 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4af25e0-6271-421d-8fce-4225d7a3f573 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:05:02.569522 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec81a43d-e1d2-43a4-a4f3-a903c0f30030 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:02.571373 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec81a43d-e1d2-43a4-a4f3-a903c0f30030 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:02.573547 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-d4af25e0-6271-421d-8fce-4225d7a3f573 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:05:02.573547 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:02.573547 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:02.573547 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:02.573547 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:02.573547 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:02.573547 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:02.573547 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:02.573547 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:02.573547 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:02.573547 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:02.573547 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:02.573547 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:05:02.573547 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:05:02.573547 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:05:02.573547 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:05:02.574502 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:05:02.574502 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:05:02.574502 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:05:02.574502 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:05:02.574502 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:02.574502 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:02.574502 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:02.574502 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:02.574502 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:02.574502 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:02.574502 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:02.574502 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:02.574502 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:02.574502 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:02.574502 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:02.574502 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:02.575301 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:02.575301 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:02.575301 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:02.575301 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:02.575301 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:05:02.575301 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:05:02.575301 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:05:02.575301 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:05:02.575301 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:05:02.575301 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:05:02.575301 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:05:02.575301 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:05:02.575301 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 56/112] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:05:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:02.576496 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d210dbe6-b105-4d6e-a793-4058b4697630 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:02.584741 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d210dbe6-b105-4d6e-a793-4058b4697630 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:02.584741 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d210dbe6-b105-4d6e-a793-4058b4697630 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:02.584741 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 57/113] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:05:02.584741 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d210dbe6-b105-4d6e-a793-4058b4697630 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:02.688882 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:02.763157 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 57/114] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:05:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:05:02.765024 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c26986-4901-4ab8-a21c-2ac70e96cb87 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:02.769133 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c26986-4901-4ab8-a21c-2ac70e96cb87 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:02.769641 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c26986-4901-4ab8-a21c-2ac70e96cb87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:02.792349 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-69c26986-4901-4ab8-a21c-2ac70e96cb87 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:02.793891 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-69c26986-4901-4ab8-a21c-2ac70e96cb87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:02.806907 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d210dbe6-b105-4d6e-a793-4058b4697630 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:02.843328 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:02.873107 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68cf0f2-a89d-4adb-b141-6e489bf1dd32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:02.875175 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68cf0f2-a89d-4adb-b141-6e489bf1dd32 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:02.875731 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68cf0f2-a89d-4adb-b141-6e489bf1dd32 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:02.876233 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68cf0f2-a89d-4adb-b141-6e489bf1dd32 None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:02.878355 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68cf0f2-a89d-4adb-b141-6e489bf1dd32 None admin] RBAC: Adding query filter params (name=member) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:02.878906 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68cf0f2-a89d-4adb-b141-6e489bf1dd32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:02.882468 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68cf0f2-a89d-4adb-b141-6e489bf1dd32 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:02.903950 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 58/115] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jun 21 18:05:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:05:02.905904 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68cf0f2-a89d-4adb-b141-6e489bf1dd32 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:02.925938 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 58/116] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:05:02.927316 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c26986-4901-4ab8-a21c-2ac70e96cb87 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:03.058468 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:03.059754 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68cf0f2-a89d-4adb-b141-6e489bf1dd32 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:03.071339 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c26986-4901-4ab8-a21c-2ac70e96cb87 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:03.106895 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617f5b48-ad93-446a-9a2c-a49d0255b6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:03.108104 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617f5b48-ad93-446a-9a2c-a49d0255b6c8 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:03.108651 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617f5b48-ad93-446a-9a2c-a49d0255b6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:03.109100 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617f5b48-ad93-446a-9a2c-a49d0255b6c8 None admin] PATH_INFO: `/v3/projects/21cd4117dcd346b8bcb3e353ff8d228a` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:03.111382 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617f5b48-ad93-446a-9a2c-a49d0255b6c8 None admin] RBAC: Authorizing `identity:get_project(project_id=21cd4117dcd346b8bcb3e353ff8d228a)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:03.114135 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617f5b48-ad93-446a-9a2c-a49d0255b6c8 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:03.115653 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf7546d-0d46-42d0-a14d-ea60534b6677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:03.116841 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf7546d-0d46-42d0-a14d-ea60534b6677 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:03.117324 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf7546d-0d46-42d0-a14d-ea60534b6677 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:03.120519 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 59/117] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jun 21 18:05:02 2022] GET /identity/v3/projects/21cd4117dcd346b8bcb3e353ff8d228a => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:03.120519 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617f5b48-ad93-446a-9a2c-a49d0255b6c8 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:03.121305 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf7546d-0d46-42d0-a14d-ea60534b6677 None admin] PATH_INFO: `/v3/users/623a005b923a4f81bdcab9453639598a` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:03.123581 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf7546d-0d46-42d0-a14d-ea60534b6677 None admin] RBAC: Authorizing `identity:get_user(user_id=623a005b923a4f81bdcab9453639598a)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:03.126364 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf7546d-0d46-42d0-a14d-ea60534b6677 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:03.129739 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 59/118] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jun 21 18:05:02 2022] GET /identity/v3/users/623a005b923a4f81bdcab9453639598a => generated 275 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:03.134142 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b8e6e-453a-40c9-9470-eba4433c7988 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:03.134142 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b8e6e-453a-40c9-9470-eba4433c7988 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:03.134142 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b8e6e-453a-40c9-9470-eba4433c7988 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:03.140729 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 60/119] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:03 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:05:03.143089 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ea0600-93ab-4e99-82ab-873892826f12 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:03.143382 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ea0600-93ab-4e99-82ab-873892826f12 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:03.143658 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ea0600-93ab-4e99-82ab-873892826f12 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:03.145550 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 61/120] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:05:03.146448 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ea0600-93ab-4e99-82ab-873892826f12 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:03.176899 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:03.235624 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617f5b48-ad93-446a-9a2c-a49d0255b6c8 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:03.272337 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ea0600-93ab-4e99-82ab-873892826f12 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:03.274027 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9655afce-1ac8-441c-89d0-9bdde8649726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:03.275155 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9655afce-1ac8-441c-89d0-9bdde8649726 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:03.275455 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9655afce-1ac8-441c-89d0-9bdde8649726 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:03.277369 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9655afce-1ac8-441c-89d0-9bdde8649726 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:03.278785 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9655afce-1ac8-441c-89d0-9bdde8649726 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:03.280273 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9655afce-1ac8-441c-89d0-9bdde8649726 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:03.282232 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 60/121] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:05:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:03.282818 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9655afce-1ac8-441c-89d0-9bdde8649726 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:03.320125 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ab81d5-9137-4a9a-928a-288f0fc48c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:03.320963 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ab81d5-9137-4a9a-928a-288f0fc48c0d None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:03.321263 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ab81d5-9137-4a9a-928a-288f0fc48c0d None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:03.321546 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ab81d5-9137-4a9a-928a-288f0fc48c0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:03.322433 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ab81d5-9137-4a9a-928a-288f0fc48c0d None admin] RBAC: Adding query filter params (role.id=55af05e4b36e44d282933b6a18958580, scope.project.id=21cd4117dcd346b8bcb3e353ff8d228a, user.id=40c2a6f1a2794b24b9a92a919a888051) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:03.323286 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ab81d5-9137-4a9a-928a-288f0fc48c0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:03.324811 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ab81d5-9137-4a9a-928a-288f0fc48c0d None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:03.331124 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 62/122] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jun 21 18:05:03 2022] GET /identity/v3/role_assignments?user.id=40c2a6f1a2794b24b9a92a919a888051&scope.project.id=21cd4117dcd346b8bcb3e353ff8d228a&role.id=55af05e4b36e44d282933b6a18958580 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:03.332448 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edaa519-b796-4ab5-acaf-926e747185f1 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:03.332748 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edaa519-b796-4ab5-acaf-926e747185f1 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:03.333050 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edaa519-b796-4ab5-acaf-926e747185f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:03.361804 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-2edaa519-b796-4ab5-acaf-926e747185f1 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:03.363517 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-2edaa519-b796-4ab5-acaf-926e747185f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:03.386508 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:03.414972 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9655afce-1ac8-441c-89d0-9bdde8649726 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:03.437302 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:03.465121 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024f36ea-6159-4d64-9b6e-0e6f1d58db0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:03.466033 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f36ea-6159-4d64-9b6e-0e6f1d58db0d None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:03.466323 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f36ea-6159-4d64-9b6e-0e6f1d58db0d None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:03.466607 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f36ea-6159-4d64-9b6e-0e6f1d58db0d None admin] PATH_INFO: `/v3/projects/243a6a54d0b6415e948330ba3cd64d7e` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:03.470149 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024f36ea-6159-4d64-9b6e-0e6f1d58db0d None admin] RBAC: Authorizing `identity:get_project(project_id=243a6a54d0b6415e948330ba3cd64d7e)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:03.474614 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024f36ea-6159-4d64-9b6e-0e6f1d58db0d None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:03.478030 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 61/123] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jun 21 18:05:03 2022] GET /identity/v3/projects/243a6a54d0b6415e948330ba3cd64d7e => generated 312 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:03.483126 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 63/124] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:05:03.483126 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb9b58e-2c38-4fde-b43a-d9d3770e2ebf None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:03.483126 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb9b58e-2c38-4fde-b43a-d9d3770e2ebf None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:03.483126 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb9b58e-2c38-4fde-b43a-d9d3770e2ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:03.484930 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024f36ea-6159-4d64-9b6e-0e6f1d58db0d None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:03.502488 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-8eb9b58e-2c38-4fde-b43a-d9d3770e2ebf None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:03.504411 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-8eb9b58e-2c38-4fde-b43a-d9d3770e2ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:03.596429 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 64/125] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:05:03.596429 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb9b58e-2c38-4fde-b43a-d9d3770e2ebf None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:03.648100 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024f36ea-6159-4d64-9b6e-0e6f1d58db0d None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:03.687242 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1c5f6d-74fa-4f76-9707-c383884470de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:03.688638 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1c5f6d-74fa-4f76-9707-c383884470de None admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:03.689155 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1c5f6d-74fa-4f76-9707-c383884470de None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:03.689638 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1c5f6d-74fa-4f76-9707-c383884470de None admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:03.690741 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1c5f6d-74fa-4f76-9707-c383884470de None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:03.693711 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1c5f6d-74fa-4f76-9707-c383884470de None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:03.706427 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:03.728712 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb9b58e-2c38-4fde-b43a-d9d3770e2ebf None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:03.771180 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e88a400-3325-4b6e-bd09-4317d6f9d97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:03.772092 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e88a400-3325-4b6e-bd09-4317d6f9d97c None admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:03.772381 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e88a400-3325-4b6e-bd09-4317d6f9d97c None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:03.772687 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e88a400-3325-4b6e-bd09-4317d6f9d97c None admin] PATH_INFO: `/v3/projects/243a6a54d0b6415e948330ba3cd64d7e/users/623a005b923a4f81bdcab9453639598a/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:03.774779 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e88a400-3325-4b6e-bd09-4317d6f9d97c None admin] RBAC: Authorizing `identity:create_grant(project_id=243a6a54d0b6415e948330ba3cd64d7e, user_id=623a005b923a4f81bdcab9453639598a, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:03.776326 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e88a400-3325-4b6e-bd09-4317d6f9d97c None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:03.817741 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 62/126] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Jun 21 18:05:03 2022] POST /identity/v3/projects => generated 302 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:05:03.825600 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1c5f6d-74fa-4f76-9707-c383884470de None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:03.887210 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 65/127] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jun 21 18:05:03 2022] PUT /identity/v3/projects/243a6a54d0b6415e948330ba3cd64d7e/users/623a005b923a4f81bdcab9453639598a/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:05:03.888447 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e88a400-3325-4b6e-bd09-4317d6f9d97c None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:03.895179 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:04.044383 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1c5f6d-74fa-4f76-9707-c383884470de None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:04.087131 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f288260b-7093-401c-be85-261a7177f38c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:04.088099 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f288260b-7093-401c-be85-261a7177f38c None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:04.088603 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f288260b-7093-401c-be85-261a7177f38c None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:04.088954 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f288260b-7093-401c-be85-261a7177f38c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:04.102631 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f288260b-7093-401c-be85-261a7177f38c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:05:04.114364 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-f288260b-7093-401c-be85-261a7177f38c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:05:04.114364 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:04.114364 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:04.114364 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:04.114364 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:04.114364 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:04.114364 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:04.114364 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:04.114364 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:04.114364 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:04.114364 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:04.114364 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:04.114364 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:05:04.114364 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:05:04.114364 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:05:04.114364 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:05:04.115501 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:05:04.115501 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:05:04.115501 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:05:04.115501 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:05:04.115501 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:04.115501 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:04.115501 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:04.115501 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:04.115501 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:04.115501 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:04.115501 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:04.115501 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:04.115501 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:04.115501 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:04.115501 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:04.115501 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:04.116606 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:04.116606 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:04.116606 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:04.116606 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:04.116606 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:05:04.116606 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:05:04.116606 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:05:04.116606 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:05:04.116606 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:05:04.116606 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:05:04.116606 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:05:04.116606 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:05:04.118401 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 63/128] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jun 21 18:05:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:04.125505 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f288260b-7093-401c-be85-261a7177f38c None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:04.178400 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e88a400-3325-4b6e-bd09-4317d6f9d97c None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:04.227777 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8725c3c4-60f1-405a-a812-808dbc94e1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:04.228643 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8725c3c4-60f1-405a-a812-808dbc94e1ec None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:04.229640 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8725c3c4-60f1-405a-a812-808dbc94e1ec None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:04.230379 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8725c3c4-60f1-405a-a812-808dbc94e1ec None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:04.241976 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:04.247064 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8725c3c4-60f1-405a-a812-808dbc94e1ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:05:04.256979 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-8725c3c4-60f1-405a-a812-808dbc94e1ec None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:05:04.256979 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:04.256979 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:04.256979 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:04.256979 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:04.256979 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:04.256979 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:04.256979 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:04.256979 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:04.256979 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:04.256979 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:04.256979 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:04.256979 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:05:04.256979 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:05:04.256979 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:05:04.256979 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:05:04.263869 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:05:04.263869 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:05:04.263869 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:05:04.263869 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:05:04.263869 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:04.263869 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:04.263869 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:04.263869 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:04.263869 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:04.263869 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:04.263869 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:04.263869 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:04.263869 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:04.263869 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:04.263869 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:04.263869 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:04.264865 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:04.264865 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:04.264865 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:04.264865 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:04.264865 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:05:04.264865 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:05:04.264865 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:05:04.264865 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:05:04.264865 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:05:04.264865 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:05:04.264865 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:05:04.264865 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:05:04.264865 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 66/129] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jun 21 18:05:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:04.267883 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8725c3c4-60f1-405a-a812-808dbc94e1ec None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:04.275050 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:04.279402 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f288260b-7093-401c-be85-261a7177f38c None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:04.322606 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2b05da-6cf1-4332-ab3f-3231e7f88767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:04.323501 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b05da-6cf1-4332-ab3f-3231e7f88767 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:04.328439 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b05da-6cf1-4332-ab3f-3231e7f88767 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:04.328439 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b05da-6cf1-4332-ab3f-3231e7f88767 None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:04.328439 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2b05da-6cf1-4332-ab3f-3231e7f88767 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:04.328439 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2b05da-6cf1-4332-ab3f-3231e7f88767 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:04.328439 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2b05da-6cf1-4332-ab3f-3231e7f88767 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:04.339172 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 64/130] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:05:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:04.349206 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2b05da-6cf1-4332-ab3f-3231e7f88767 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:04.465652 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8725c3c4-60f1-405a-a812-808dbc94e1ec None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:04.474790 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2b05da-6cf1-4332-ab3f-3231e7f88767 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:04.518399 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7798433d-832e-44a3-b57a-7e0ddda15086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:04.519386 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7798433d-832e-44a3-b57a-7e0ddda15086 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:04.519747 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7798433d-832e-44a3-b57a-7e0ddda15086 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:04.519904 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50914e32-5d5b-40ae-95a7-a2d51d5f01ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:04.520374 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7798433d-832e-44a3-b57a-7e0ddda15086 None admin] PATH_INFO: `/v3/users/0c89ac265bd34c75a692d91d672fad59` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:04.521215 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50914e32-5d5b-40ae-95a7-a2d51d5f01ee None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:04.521564 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7798433d-832e-44a3-b57a-7e0ddda15086 None admin] RBAC: Authorizing `identity:get_user(user_id=0c89ac265bd34c75a692d91d672fad59)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:04.521791 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50914e32-5d5b-40ae-95a7-a2d51d5f01ee None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:04.522217 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50914e32-5d5b-40ae-95a7-a2d51d5f01ee None admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:04.523184 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50914e32-5d5b-40ae-95a7-a2d51d5f01ee None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:04.523294 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7798433d-832e-44a3-b57a-7e0ddda15086 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:04.523728 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50914e32-5d5b-40ae-95a7-a2d51d5f01ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:04.528426 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50914e32-5d5b-40ae-95a7-a2d51d5f01ee None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:04.528576 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 65/131] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jun 21 18:05:04 2022] GET /identity/v3/users/0c89ac265bd34c75a692d91d672fad59 => generated 247 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:04.533873 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7798433d-832e-44a3-b57a-7e0ddda15086 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:04.540448 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 67/132] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jun 21 18:05:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:05:04.549480 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50914e32-5d5b-40ae-95a7-a2d51d5f01ee None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:04.572021 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:04.665194 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7798433d-832e-44a3-b57a-7e0ddda15086 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:04.678315 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:04.707689 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf16324-0bd8-45b6-94bf-1617de590f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:04.708849 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf16324-0bd8-45b6-94bf-1617de590f8e None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:04.709288 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf16324-0bd8-45b6-94bf-1617de590f8e None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:04.709716 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf16324-0bd8-45b6-94bf-1617de590f8e None admin] PATH_INFO: `/v3/projects/21cd4117dcd346b8bcb3e353ff8d228a` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:04.717458 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf16324-0bd8-45b6-94bf-1617de590f8e None admin] RBAC: Authorizing `identity:get_project(project_id=21cd4117dcd346b8bcb3e353ff8d228a)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:04.719197 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf16324-0bd8-45b6-94bf-1617de590f8e None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:04.721194 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 66/133] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jun 21 18:05:04 2022] GET /identity/v3/projects/21cd4117dcd346b8bcb3e353ff8d228a => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:05:04.723357 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87f4b1d-1ac7-4450-9ee2-577969800aa1 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:04.726912 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87f4b1d-1ac7-4450-9ee2-577969800aa1 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:04.728247 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87f4b1d-1ac7-4450-9ee2-577969800aa1 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:04.730447 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 67/134] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:04 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:05:04.733879 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87f4b1d-1ac7-4450-9ee2-577969800aa1 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:04.764705 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50914e32-5d5b-40ae-95a7-a2d51d5f01ee None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:04.835345 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b8431f-7ac7-40fe-8633-58dacf7c7e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:04.837105 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b8431f-7ac7-40fe-8633-58dacf7c7e8c None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:04.837578 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b8431f-7ac7-40fe-8633-58dacf7c7e8c None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:04.838131 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b8431f-7ac7-40fe-8633-58dacf7c7e8c None admin] PATH_INFO: `/v3/users/623a005b923a4f81bdcab9453639598a` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:04.840282 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b8431f-7ac7-40fe-8633-58dacf7c7e8c None admin] RBAC: Authorizing `identity:get_user(user_id=623a005b923a4f81bdcab9453639598a)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:04.842246 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b8431f-7ac7-40fe-8633-58dacf7c7e8c None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:04.844103 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 68/135] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jun 21 18:05:04 2022] GET /identity/v3/users/623a005b923a4f81bdcab9453639598a => generated 275 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:04.845993 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94651be-78ae-436a-b31f-f6481fddfb98 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:04.846496 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94651be-78ae-436a-b31f-f6481fddfb98 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:04.846924 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94651be-78ae-436a-b31f-f6481fddfb98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:04.866484 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-a94651be-78ae-436a-b31f-f6481fddfb98 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:04.868738 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-a94651be-78ae-436a-b31f-f6481fddfb98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:04.957801 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:04.965836 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87f4b1d-1ac7-4450-9ee2-577969800aa1 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:04.981077 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 69/136] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:05:04.982811 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94651be-78ae-436a-b31f-f6481fddfb98 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:05.042544 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78773e7-0bff-4661-a064-286a3961453b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:05.043346 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78773e7-0bff-4661-a064-286a3961453b None admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:05.043637 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78773e7-0bff-4661-a064-286a3961453b None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:05.043972 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78773e7-0bff-4661-a064-286a3961453b None admin] PATH_INFO: `/v3/projects/21cd4117dcd346b8bcb3e353ff8d228a/users/0c89ac265bd34c75a692d91d672fad59/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:05.050734 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78773e7-0bff-4661-a064-286a3961453b None admin] RBAC: Authorizing `identity:create_grant(project_id=21cd4117dcd346b8bcb3e353ff8d228a, user_id=0c89ac265bd34c75a692d91d672fad59, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:05.052595 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78773e7-0bff-4661-a064-286a3961453b None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:05.061168 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:05.108157 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94651be-78ae-436a-b31f-f6481fddfb98 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:05.145478 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96ae780-0560-4b86-b8e7-10d906d72b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:05.146860 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96ae780-0560-4b86-b8e7-10d906d72b0c None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:05.147367 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96ae780-0560-4b86-b8e7-10d906d72b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:05.147884 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96ae780-0560-4b86-b8e7-10d906d72b0c None admin] PATH_INFO: `/v3/projects/21cd4117dcd346b8bcb3e353ff8d228a` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:05.149409 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96ae780-0560-4b86-b8e7-10d906d72b0c None admin] RBAC: Authorizing `identity:get_project(project_id=21cd4117dcd346b8bcb3e353ff8d228a)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:05.151722 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96ae780-0560-4b86-b8e7-10d906d72b0c None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:05.153776 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 70/137] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jun 21 18:05:04 2022] GET /identity/v3/projects/21cd4117dcd346b8bcb3e353ff8d228a => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:05.154841 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96ae780-0560-4b86-b8e7-10d906d72b0c None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:05.169727 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 68/138] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jun 21 18:05:04 2022] PUT /identity/v3/projects/21cd4117dcd346b8bcb3e353ff8d228a/users/0c89ac265bd34c75a692d91d672fad59/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:05:05.170710 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78773e7-0bff-4661-a064-286a3961453b None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:05.273679 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96ae780-0560-4b86-b8e7-10d906d72b0c None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:05.305607 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:05.314995 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d633c-d90d-42cc-bf7e-4b00de6c125e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:05.317854 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d633c-d90d-42cc-bf7e-4b00de6c125e None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:05.317854 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d633c-d90d-42cc-bf7e-4b00de6c125e None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:05.317854 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d633c-d90d-42cc-bf7e-4b00de6c125e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:05.331955 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6d633c-d90d-42cc-bf7e-4b00de6c125e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:05:05.347764 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-9c6d633c-d90d-42cc-bf7e-4b00de6c125e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:05:05.347764 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:05.347764 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:05.347764 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:05.347764 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:05.347764 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:05.347764 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:05.347764 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:05.347764 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:05.347764 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:05.347764 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:05.347764 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:05.347764 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:05:05.347764 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:05:05.347764 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:05:05.347764 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:05:05.348982 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:05:05.348982 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:05:05.348982 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:05:05.348982 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:05:05.348982 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:05.348982 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:05.348982 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:05.348982 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:05.348982 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:05.348982 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:05.348982 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:05.348982 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:05.348982 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:05.348982 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:05.348982 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:05.348982 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:05.349986 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:05.349986 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:05.349986 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:05.349986 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:05.349986 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:05:05.349986 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:05:05.349986 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:05:05.349986 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:05:05.349986 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:05:05.349986 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:05:05.349986 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:05:05.349986 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:05:05.351561 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 71/139] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:05:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:05.361904 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d633c-d90d-42cc-bf7e-4b00de6c125e None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:05.466421 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78773e7-0bff-4661-a064-286a3961453b None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:05.568187 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d633c-d90d-42cc-bf7e-4b00de6c125e None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:05.603201 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5352ac6-4664-4687-b054-d039a11e0ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:05.604609 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5352ac6-4664-4687-b054-d039a11e0ab7 None admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:05.604905 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5352ac6-4664-4687-b054-d039a11e0ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:05.605194 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5352ac6-4664-4687-b054-d039a11e0ab7 None admin] PATH_INFO: `/v3/projects/21cd4117dcd346b8bcb3e353ff8d228a/users/623a005b923a4f81bdcab9453639598a/roles/482dcc9ba0fd400cad2d193e3e895d4c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:05.607693 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5352ac6-4664-4687-b054-d039a11e0ab7 None admin] RBAC: Authorizing `identity:create_grant(project_id=21cd4117dcd346b8bcb3e353ff8d228a, user_id=623a005b923a4f81bdcab9453639598a, role_id=482dcc9ba0fd400cad2d193e3e895d4c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:05.612519 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5352ac6-4664-4687-b054-d039a11e0ab7 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:05.638672 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9061e2f-357a-4725-9e87-c60e0618aaba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:05.639387 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9061e2f-357a-4725-9e87-c60e0618aaba None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:05.639673 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9061e2f-357a-4725-9e87-c60e0618aaba None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:05.639954 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9061e2f-357a-4725-9e87-c60e0618aaba None admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:05.640780 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9061e2f-357a-4725-9e87-c60e0618aaba None admin] RBAC: Adding query filter params (name=member) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:05.641093 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9061e2f-357a-4725-9e87-c60e0618aaba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:05.642544 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9061e2f-357a-4725-9e87-c60e0618aaba None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:05.652735 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 72/140] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jun 21 18:05:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:05.658972 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9061e2f-357a-4725-9e87-c60e0618aaba None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:05.703640 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 69/141] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jun 21 18:05:05 2022] PUT /identity/v3/projects/21cd4117dcd346b8bcb3e353ff8d228a/users/623a005b923a4f81bdcab9453639598a/roles/482dcc9ba0fd400cad2d193e3e895d4c => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:05:05.749287 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:05.820798 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9061e2f-357a-4725-9e87-c60e0618aaba None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:05.860177 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17b0268-9d80-4aee-a640-3e329da42e6d None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:05.860594 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17b0268-9d80-4aee-a640-3e329da42e6d None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:05.860982 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17b0268-9d80-4aee-a640-3e329da42e6d None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:05.861514 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7516931c-5fff-4756-b630-d3a982e7340a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:05.862522 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 70/142] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:05:05.862630 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7516931c-5fff-4756-b630-d3a982e7340a None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:05.862811 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7516931c-5fff-4756-b630-d3a982e7340a None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:05.863044 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7516931c-5fff-4756-b630-d3a982e7340a None admin] PATH_INFO: `/v3/users/623a005b923a4f81bdcab9453639598a` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:05.869295 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7516931c-5fff-4756-b630-d3a982e7340a None admin] RBAC: Authorizing `identity:get_user(user_id=623a005b923a4f81bdcab9453639598a)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:05.872140 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7516931c-5fff-4756-b630-d3a982e7340a None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:05.873912 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50721968-a084-4146-aadb-64e7dcd0957b None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:05.874210 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50721968-a084-4146-aadb-64e7dcd0957b None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:05.874401 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 73/143] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jun 21 18:05:05 2022] GET /identity/v3/users/623a005b923a4f81bdcab9453639598a => generated 275 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:05.874549 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50721968-a084-4146-aadb-64e7dcd0957b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:05.879418 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7516931c-5fff-4756-b630-d3a982e7340a None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:05.909773 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-50721968-a084-4146-aadb-64e7dcd0957b None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:05.911021 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-50721968-a084-4146-aadb-64e7dcd0957b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:05.932852 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:06.070638 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7516931c-5fff-4756-b630-d3a982e7340a None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:06.089344 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 71/144] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:05:06.114127 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:06.140027 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f307256b-b696-4778-bed8-da151d37d231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:06.140728 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f307256b-b696-4778-bed8-da151d37d231 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:06.141015 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f307256b-b696-4778-bed8-da151d37d231 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:06.141295 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f307256b-b696-4778-bed8-da151d37d231 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:06.156454 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50721968-a084-4146-aadb-64e7dcd0957b None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:06.279650 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f307256b-b696-4778-bed8-da151d37d231 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:06.281910 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f307256b-b696-4778-bed8-da151d37d231 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:06.410669 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50721968-a084-4146-aadb-64e7dcd0957b None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:06.462348 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 74/145] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:05:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 583 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 21 18:05:06.469816 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f307256b-b696-4778-bed8-da151d37d231 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:06.504142 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:06.559359 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18912284-d9a7-4033-81c6-c2757ca45f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:06.560558 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18912284-d9a7-4033-81c6-c2757ca45f64 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:06.560558 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18912284-d9a7-4033-81c6-c2757ca45f64 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:06.560840 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18912284-d9a7-4033-81c6-c2757ca45f64 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:06.586421 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18912284-d9a7-4033-81c6-c2757ca45f64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:05:06.600154 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-18912284-d9a7-4033-81c6-c2757ca45f64 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 18:05:06.600154 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:06.600154 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:06.600154 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:06.600154 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:06.600154 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:06.600154 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:06.600154 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:06.600154 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:06.600154 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:06.600154 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:06.600154 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:06.600154 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:05:06.600154 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:05:06.600154 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:05:06.600154 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:05:06.601410 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:05:06.601410 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:05:06.601410 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:05:06.601410 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:05:06.601410 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:06.601410 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:06.601410 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:06.601410 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:06.601410 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:06.601410 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:06.601410 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:06.601410 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:06.601410 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:06.601410 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:06.601410 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:06.601410 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:06.602253 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:06.602253 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:06.602253 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:06.602253 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:06.602253 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:05:06.602253 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:05:06.602253 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:05:06.602253 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:05:06.602253 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:05:06.602253 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:05:06.602253 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 18:05:06.602253 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:05:06.602253 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 72/146] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:05:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 451 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:05:06.613056 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18912284-d9a7-4033-81c6-c2757ca45f64 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:06.631280 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:06.650564 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f307256b-b696-4778-bed8-da151d37d231 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:06.731588 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5e9b40-b0c6-4f0c-a054-c01468b84e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:06.731588 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5e9b40-b0c6-4f0c-a054-c01468b84e06 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:06.731588 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5e9b40-b0c6-4f0c-a054-c01468b84e06 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:06.731588 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5e9b40-b0c6-4f0c-a054-c01468b84e06 None admin] PATH_INFO: `/v3/projects/21cd4117dcd346b8bcb3e353ff8d228a` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:06.732151 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5e9b40-b0c6-4f0c-a054-c01468b84e06 None admin] RBAC: Authorizing `identity:get_project(project_id=21cd4117dcd346b8bcb3e353ff8d228a)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:06.733670 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5e9b40-b0c6-4f0c-a054-c01468b84e06 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:06.735468 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 75/147] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jun 21 18:05:06 2022] GET /identity/v3/projects/21cd4117dcd346b8bcb3e353ff8d228a => generated 298 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:06.752557 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5e9b40-b0c6-4f0c-a054-c01468b84e06 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:06.837241 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18912284-d9a7-4033-81c6-c2757ca45f64 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:06.883822 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:06.900170 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5e9b40-b0c6-4f0c-a054-c01468b84e06 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:06.937807 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673f447f-ab2a-4995-b3d2-21667312935e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:06.938341 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b797b3b-c00f-48ea-a4d2-457ad0076584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:06.938541 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f447f-ab2a-4995-b3d2-21667312935e None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:06.938800 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f447f-ab2a-4995-b3d2-21667312935e None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:06.939025 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b797b3b-c00f-48ea-a4d2-457ad0076584 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:06.939125 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f447f-ab2a-4995-b3d2-21667312935e None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:06.939313 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b797b3b-c00f-48ea-a4d2-457ad0076584 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:06.939823 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673f447f-ab2a-4995-b3d2-21667312935e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:06.939992 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b797b3b-c00f-48ea-a4d2-457ad0076584 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:06.940153 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673f447f-ab2a-4995-b3d2-21667312935e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:06.940883 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b797b3b-c00f-48ea-a4d2-457ad0076584 None admin] RBAC: Adding query filter params (role.id=c467f10d3dba4f2dbfd42068f42f74b6, scope.project.id=21cd4117dcd346b8bcb3e353ff8d228a, user.id=623a005b923a4f81bdcab9453639598a) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:06.941195 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b797b3b-c00f-48ea-a4d2-457ad0076584 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:06.941578 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673f447f-ab2a-4995-b3d2-21667312935e None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:06.942743 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b797b3b-c00f-48ea-a4d2-457ad0076584 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:06.956615 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 76/148] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jun 21 18:05:06 2022] GET /identity/v3/role_assignments?user.id=623a005b923a4f81bdcab9453639598a&scope.project.id=21cd4117dcd346b8bcb3e353ff8d228a&role.id=c467f10d3dba4f2dbfd42068f42f74b6 => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:06.958270 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd26a4-0669-4ad2-bb34-591c5a80de7d None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:06.958564 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd26a4-0669-4ad2-bb34-591c5a80de7d None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:06.958844 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd26a4-0669-4ad2-bb34-591c5a80de7d None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:06.959671 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 73/149] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jun 21 18:05:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:06.960221 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 77/150] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:05:06.966737 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673f447f-ab2a-4995-b3d2-21667312935e None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:06.967178 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a2af67-dc34-4f69-a39f-ec2d14049fb2 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:06.967458 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a2af67-dc34-4f69-a39f-ec2d14049fb2 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:06.967740 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a2af67-dc34-4f69-a39f-ec2d14049fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:07.001732 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-54a2af67-dc34-4f69-a39f-ec2d14049fb2 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:07.002902 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-54a2af67-dc34-4f69-a39f-ec2d14049fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:07.118915 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 78/151] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:05:07.167702 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a2af67-dc34-4f69-a39f-ec2d14049fb2 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:07.180741 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:07.259744 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673f447f-ab2a-4995-b3d2-21667312935e None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:07.333301 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea60d1d-b63e-4286-af50-4fef7c443f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:07.334260 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea60d1d-b63e-4286-af50-4fef7c443f5c None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:07.334679 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea60d1d-b63e-4286-af50-4fef7c443f5c None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:07.334959 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea60d1d-b63e-4286-af50-4fef7c443f5c None admin] PATH_INFO: `/v3/users/40c2a6f1a2794b24b9a92a919a888051` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:07.336456 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea60d1d-b63e-4286-af50-4fef7c443f5c None admin] RBAC: Authorizing `identity:get_user(user_id=40c2a6f1a2794b24b9a92a919a888051)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:07.338327 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea60d1d-b63e-4286-af50-4fef7c443f5c None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:07.341555 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 74/152] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jun 21 18:05:06 2022] GET /identity/v3/users/40c2a6f1a2794b24b9a92a919a888051 => generated 289 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:07.343159 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f411b1b-3558-4bf3-82ed-dd80da5e4140 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:07.343441 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f411b1b-3558-4bf3-82ed-dd80da5e4140 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:07.343753 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f411b1b-3558-4bf3-82ed-dd80da5e4140 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:07.345562 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 75/153] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:05:07.352206 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f411b1b-3558-4bf3-82ed-dd80da5e4140 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:07.354939 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:07.400384 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a2af67-dc34-4f69-a39f-ec2d14049fb2 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:07.501045 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c7ef05-fbe0-41bd-b772-001b4fb3d2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:07.501863 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c7ef05-fbe0-41bd-b772-001b4fb3d2ed None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:07.502146 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c7ef05-fbe0-41bd-b772-001b4fb3d2ed None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:07.502419 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c7ef05-fbe0-41bd-b772-001b4fb3d2ed None admin] PATH_INFO: `/v3/domains/default` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:07.503653 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c7ef05-fbe0-41bd-b772-001b4fb3d2ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:07.505159 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c7ef05-fbe0-41bd-b772-001b4fb3d2ed None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:07.507022 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 79/154] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:05:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:05:07.508205 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ec529c-0c37-4dde-ba44-07d1840ec599 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:07.508594 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ec529c-0c37-4dde-ba44-07d1840ec599 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:07.508876 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ec529c-0c37-4dde-ba44-07d1840ec599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:07.519994 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f411b1b-3558-4bf3-82ed-dd80da5e4140 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:07.529967 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-96ec529c-0c37-4dde-ba44-07d1840ec599 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:07.531192 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-96ec529c-0c37-4dde-ba44-07d1840ec599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:07.580013 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f97fded-7992-4214-a7fe-06eaf24a5a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:07.580994 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f97fded-7992-4214-a7fe-06eaf24a5a5a None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:07.581286 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f97fded-7992-4214-a7fe-06eaf24a5a5a None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:07.581563 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f97fded-7992-4214-a7fe-06eaf24a5a5a None admin] PATH_INFO: `/v3/projects/21cd4117dcd346b8bcb3e353ff8d228a` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:07.584998 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f97fded-7992-4214-a7fe-06eaf24a5a5a None admin] RBAC: Authorizing `identity:get_project(project_id=21cd4117dcd346b8bcb3e353ff8d228a)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:07.586616 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f97fded-7992-4214-a7fe-06eaf24a5a5a None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:07.588889 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 76/155] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jun 21 18:05:07 2022] GET /identity/v3/projects/21cd4117dcd346b8bcb3e353ff8d228a => generated 298 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:07.589428 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f97fded-7992-4214-a7fe-06eaf24a5a5a None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:07.593728 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:07.638338 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 80/156] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:05:07.640205 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ec529c-0c37-4dde-ba44-07d1840ec599 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:07.714580 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f97fded-7992-4214-a7fe-06eaf24a5a5a None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:07.751988 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3aef08-bc64-4ddf-bf52-afb562014933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:07.752707 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3aef08-bc64-4ddf-bf52-afb562014933 None admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:07.752990 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3aef08-bc64-4ddf-bf52-afb562014933 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:07.753261 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3aef08-bc64-4ddf-bf52-afb562014933 None admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:07.753989 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3aef08-bc64-4ddf-bf52-afb562014933 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:07.755437 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3aef08-bc64-4ddf-bf52-afb562014933 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:07.790550 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:07.828908 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ec529c-0c37-4dde-ba44-07d1840ec599 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:07.881019 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ac0113-e0bc-438f-8f4f-5c950d68bbbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:07.882731 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ac0113-e0bc-438f-8f4f-5c950d68bbbd None admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:07.882731 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ac0113-e0bc-438f-8f4f-5c950d68bbbd None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:07.882731 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ac0113-e0bc-438f-8f4f-5c950d68bbbd None admin] PATH_INFO: `/v3/projects/21cd4117dcd346b8bcb3e353ff8d228a/users/40c2a6f1a2794b24b9a92a919a888051/roles/55af05e4b36e44d282933b6a18958580` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:07.889903 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ac0113-e0bc-438f-8f4f-5c950d68bbbd None admin] RBAC: Authorizing `identity:create_grant(project_id=21cd4117dcd346b8bcb3e353ff8d228a, user_id=40c2a6f1a2794b24b9a92a919a888051, role_id=55af05e4b36e44d282933b6a18958580)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:07.893386 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 77/157] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Jun 21 18:05:07 2022] POST /identity/v3/users => generated 283 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:05:07.894061 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ac0113-e0bc-438f-8f4f-5c950d68bbbd None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:07.902314 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3aef08-bc64-4ddf-bf52-afb562014933 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:07.990351 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 81/158] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jun 21 18:05:07 2022] PUT /identity/v3/projects/21cd4117dcd346b8bcb3e353ff8d228a/users/40c2a6f1a2794b24b9a92a919a888051/roles/55af05e4b36e44d282933b6a18958580 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:05:08.017962 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:08.085730 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3aef08-bc64-4ddf-bf52-afb562014933 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:08.146120 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c83aad2-8bac-4f5d-b924-d8ae133e25fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:08.147291 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c83aad2-8bac-4f5d-b924-d8ae133e25fe None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:08.147772 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c83aad2-8bac-4f5d-b924-d8ae133e25fe None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:08.148230 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c83aad2-8bac-4f5d-b924-d8ae133e25fe None admin] PATH_INFO: `/v3/roles/member` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:08.162578 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c83aad2-8bac-4f5d-b924-d8ae133e25fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:05:08.171761 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-1c83aad2-8bac-4f5d-b924-d8ae133e25fe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:05:08.171761 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:08.171761 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:08.171761 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:08.171761 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:08.171761 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:08.171761 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:08.171761 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:08.171761 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:08.171761 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:08.171761 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:08.171761 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:08.171761 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:05:08.171761 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:05:08.171761 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:05:08.171761 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:05:08.172806 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:05:08.172806 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:05:08.172806 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:05:08.172806 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:05:08.172806 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:08.172806 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:08.172806 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:08.172806 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:08.172806 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:08.172806 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:08.172806 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:08.172806 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:08.172806 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:08.172806 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:08.172806 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:08.172806 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:08.173737 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:08.173737 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:08.173737 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:08.173737 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:08.173737 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:05:08.173737 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:05:08.173737 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:05:08.173737 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:05:08.173737 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:05:08.173737 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:05:08.173737 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:05:08.173737 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:05:08.176061 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 78/159] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:05:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:08.183887 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ac0113-e0bc-438f-8f4f-5c950d68bbbd None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:08.314467 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ac0113-e0bc-438f-8f4f-5c950d68bbbd None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:08.352691 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777b64d8-04b9-4c3f-91f9-76cb5cce0c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:08.353305 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777b64d8-04b9-4c3f-91f9-76cb5cce0c0f None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:08.353589 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777b64d8-04b9-4c3f-91f9-76cb5cce0c0f None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:08.354320 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777b64d8-04b9-4c3f-91f9-76cb5cce0c0f None admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:08.355146 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777b64d8-04b9-4c3f-91f9-76cb5cce0c0f None admin] RBAC: Adding query filter params (name=member) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:08.355462 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777b64d8-04b9-4c3f-91f9-76cb5cce0c0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:08.356898 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777b64d8-04b9-4c3f-91f9-76cb5cce0c0f None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:08.362831 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:08.366760 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 82/160] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jun 21 18:05:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:08.373015 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c83aad2-8bac-4f5d-b924-d8ae133e25fe None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:08.514361 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c83aad2-8bac-4f5d-b924-d8ae133e25fe None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:08.543182 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:08.561983 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff14c71-83b9-420f-b9c2-d30ee1e95ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:08.562905 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff14c71-83b9-420f-b9c2-d30ee1e95ccf None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:08.563195 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff14c71-83b9-420f-b9c2-d30ee1e95ccf None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:08.563478 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff14c71-83b9-420f-b9c2-d30ee1e95ccf None admin] PATH_INFO: `/v3/users/623a005b923a4f81bdcab9453639598a` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:08.564921 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff14c71-83b9-420f-b9c2-d30ee1e95ccf None admin] RBAC: Authorizing `identity:get_user(user_id=623a005b923a4f81bdcab9453639598a)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:08.566563 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff14c71-83b9-420f-b9c2-d30ee1e95ccf None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:08.568338 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 79/161] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jun 21 18:05:08 2022] GET /identity/v3/users/623a005b923a4f81bdcab9453639598a => generated 275 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:08.578363 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777b64d8-04b9-4c3f-91f9-76cb5cce0c0f None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:08.648222 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e357a-8a71-420b-b2da-0c120053aabe None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:08.648447 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e357a-8a71-420b-b2da-0c120053aabe None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:08.648741 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e357a-8a71-420b-b2da-0c120053aabe None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:08.650197 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 80/162] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:05:08.656730 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9890b2bf-d8f5-4ef9-82ac-03ccbcbecef0 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:08.657354 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9890b2bf-d8f5-4ef9-82ac-03ccbcbecef0 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:08.657836 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9890b2bf-d8f5-4ef9-82ac-03ccbcbecef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:08.693563 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-9890b2bf-d8f5-4ef9-82ac-03ccbcbecef0 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:08.695703 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-9890b2bf-d8f5-4ef9-82ac-03ccbcbecef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:08.736425 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777b64d8-04b9-4c3f-91f9-76cb5cce0c0f None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:08.785026 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c38d45-8e76-42cd-9b56-1c4a3dcb1ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:08.785702 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c38d45-8e76-42cd-9b56-1c4a3dcb1ce3 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:08.785991 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c38d45-8e76-42cd-9b56-1c4a3dcb1ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:08.786271 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c38d45-8e76-42cd-9b56-1c4a3dcb1ce3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:08.796902 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 81/163] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:05:08.798291 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7bd96-0121-4361-9c64-bc45d695dba3 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:08.799102 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7bd96-0121-4361-9c64-bc45d695dba3 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:08.799393 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7bd96-0121-4361-9c64-bc45d695dba3 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:08.801231 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 82/164] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:05:08.807510 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74967a8-ff2b-44a0-8de2-2a29746ba271 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:08.807806 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74967a8-ff2b-44a0-8de2-2a29746ba271 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:08.808087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74967a8-ff2b-44a0-8de2-2a29746ba271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:08.831098 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-b74967a8-ff2b-44a0-8de2-2a29746ba271 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:08.832605 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-b74967a8-ff2b-44a0-8de2-2a29746ba271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:08.888782 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:08.892182 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c38d45-8e76-42cd-9b56-1c4a3dcb1ce3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:08.893788 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c38d45-8e76-42cd-9b56-1c4a3dcb1ce3 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:08.923174 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 83/165] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:05:08.924027 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74967a8-ff2b-44a0-8de2-2a29746ba271 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:09.013676 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:09.037736 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 83/166] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:05:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:05:09.040224 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c38d45-8e76-42cd-9b56-1c4a3dcb1ce3 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:09.081354 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74967a8-ff2b-44a0-8de2-2a29746ba271 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:09.142687 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dfceba-9376-44dc-88db-5b646c682e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:09.144117 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dfceba-9376-44dc-88db-5b646c682e01 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:09.144631 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dfceba-9376-44dc-88db-5b646c682e01 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:09.145087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dfceba-9376-44dc-88db-5b646c682e01 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:09.167429 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dfceba-9376-44dc-88db-5b646c682e01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:05:09.172692 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c38d45-8e76-42cd-9b56-1c4a3dcb1ce3 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:09.174928 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:09.183482 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-42dfceba-9376-44dc-88db-5b646c682e01 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:05:09.183482 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:09.183482 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:09.183482 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:09.183482 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:09.183482 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:09.183482 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:09.183482 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:09.183482 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:09.183482 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:09.183482 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:09.183482 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:09.183482 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:05:09.183482 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:05:09.183482 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:05:09.183482 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:05:09.184739 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:05:09.184739 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:05:09.184739 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:05:09.184739 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:05:09.184739 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:09.184739 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:09.184739 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:09.184739 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:09.184739 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:09.184739 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:09.184739 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:09.184739 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:09.184739 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:09.184739 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:09.184739 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:09.184739 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:09.185984 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:09.185984 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:09.185984 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:09.185984 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:09.185984 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:05:09.185984 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:05:09.185984 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:05:09.185984 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:05:09.185984 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:05:09.185984 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:05:09.185984 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:05:09.185984 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:05:09.185984 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 84/167] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jun 21 18:05:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:09.185984 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dfceba-9376-44dc-88db-5b646c682e01 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:09.235258 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe4419-f938-4705-9309-75cf71d06f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:09.236370 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe4419-f938-4705-9309-75cf71d06f0e None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:09.236826 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe4419-f938-4705-9309-75cf71d06f0e None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:09.237256 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe4419-f938-4705-9309-75cf71d06f0e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:09.263642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefe4419-f938-4705-9309-75cf71d06f0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:05:09.271695 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-cefe4419-f938-4705-9309-75cf71d06f0e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:05:09.271695 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:09.271695 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:09.271695 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:09.271695 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:09.271695 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:09.271695 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:09.271695 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:09.271695 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:09.271695 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:09.271695 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:09.271695 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:09.271695 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:05:09.271695 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:05:09.271695 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:05:09.271695 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:05:09.272891 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:05:09.272891 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:05:09.272891 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:05:09.272891 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:05:09.272891 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:09.272891 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:09.272891 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:09.272891 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:09.272891 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:09.272891 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:09.272891 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:09.272891 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:09.272891 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:09.272891 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:09.272891 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:09.272891 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:09.273868 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:09.273868 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:09.273868 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:09.273868 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:09.273868 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:05:09.273868 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:05:09.273868 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:05:09.273868 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:05:09.273868 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:05:09.273868 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:05:09.273868 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:05:09.273868 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:05:09.280416 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 84/168] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jun 21 18:05:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:09.281632 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe4419-f938-4705-9309-75cf71d06f0e None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:09.314694 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dfceba-9376-44dc-88db-5b646c682e01 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:09.325091 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:09.353282 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecadc6e-a57b-4b6c-a2b4-8513be48b851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:09.354941 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecadc6e-a57b-4b6c-a2b4-8513be48b851 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:09.355226 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecadc6e-a57b-4b6c-a2b4-8513be48b851 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:09.355501 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecadc6e-a57b-4b6c-a2b4-8513be48b851 None admin] PATH_INFO: `/v3/projects/243a6a54d0b6415e948330ba3cd64d7e` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:09.356806 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecadc6e-a57b-4b6c-a2b4-8513be48b851 None admin] RBAC: Authorizing `identity:get_project(project_id=243a6a54d0b6415e948330ba3cd64d7e)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:09.358261 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecadc6e-a57b-4b6c-a2b4-8513be48b851 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:09.360305 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 85/169] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jun 21 18:05:09 2022] GET /identity/v3/projects/243a6a54d0b6415e948330ba3cd64d7e => generated 312 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:09.360996 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecadc6e-a57b-4b6c-a2b4-8513be48b851 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:09.474941 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecadc6e-a57b-4b6c-a2b4-8513be48b851 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:09.525129 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45a654d-bffc-4617-8bf3-5ad51fba286f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:09.526554 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45a654d-bffc-4617-8bf3-5ad51fba286f None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:09.526841 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45a654d-bffc-4617-8bf3-5ad51fba286f None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:09.527124 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45a654d-bffc-4617-8bf3-5ad51fba286f None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:09.527943 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45a654d-bffc-4617-8bf3-5ad51fba286f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:09.528255 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45a654d-bffc-4617-8bf3-5ad51fba286f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:09.529690 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45a654d-bffc-4617-8bf3-5ad51fba286f None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:09.538988 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 86/170] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:05:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:09.539952 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45a654d-bffc-4617-8bf3-5ad51fba286f None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:09.590141 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:09.599044 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe4419-f938-4705-9309-75cf71d06f0e None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:09.658147 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404740ca-218f-4402-a4e5-936cb2538c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:09.659300 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404740ca-218f-4402-a4e5-936cb2538c32 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:09.659786 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404740ca-218f-4402-a4e5-936cb2538c32 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:09.660258 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404740ca-218f-4402-a4e5-936cb2538c32 None admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:09.661338 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404740ca-218f-4402-a4e5-936cb2538c32 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:09.661889 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404740ca-218f-4402-a4e5-936cb2538c32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:09.664689 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404740ca-218f-4402-a4e5-936cb2538c32 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:09.675324 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 85/171] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jun 21 18:05:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:09.676514 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404740ca-218f-4402-a4e5-936cb2538c32 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:09.709321 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:09.723300 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45a654d-bffc-4617-8bf3-5ad51fba286f None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:09.803396 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3127657-3c05-4e57-87ca-7b5d8d5b9064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:09.804444 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3127657-3c05-4e57-87ca-7b5d8d5b9064 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:09.804932 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3127657-3c05-4e57-87ca-7b5d8d5b9064 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:09.805406 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3127657-3c05-4e57-87ca-7b5d8d5b9064 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:09.806470 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3127657-3c05-4e57-87ca-7b5d8d5b9064 None admin] RBAC: Adding query filter params (role.id=c467f10d3dba4f2dbfd42068f42f74b6, scope.project.id=243a6a54d0b6415e948330ba3cd64d7e, user.id=623a005b923a4f81bdcab9453639598a) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:09.809178 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3127657-3c05-4e57-87ca-7b5d8d5b9064 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:09.810940 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3127657-3c05-4e57-87ca-7b5d8d5b9064 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:09.824702 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 87/172] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jun 21 18:05:09 2022] GET /identity/v3/role_assignments?user.id=623a005b923a4f81bdcab9453639598a&scope.project.id=243a6a54d0b6415e948330ba3cd64d7e&role.id=c467f10d3dba4f2dbfd42068f42f74b6 => generated 608 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:09.826555 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3127657-3c05-4e57-87ca-7b5d8d5b9064 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:09.874355 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404740ca-218f-4402-a4e5-936cb2538c32 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:09.969622 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c85cb4-f06c-4558-ab4a-4280a036eb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:09.971421 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c85cb4-f06c-4558-ab4a-4280a036eb98 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:09.971956 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c85cb4-f06c-4558-ab4a-4280a036eb98 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:09.972715 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c85cb4-f06c-4558-ab4a-4280a036eb98 None admin] PATH_INFO: `/v3/users/0c89ac265bd34c75a692d91d672fad59` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:09.977324 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c85cb4-f06c-4558-ab4a-4280a036eb98 None admin] RBAC: Authorizing `identity:get_user(user_id=0c89ac265bd34c75a692d91d672fad59)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:09.979482 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c85cb4-f06c-4558-ab4a-4280a036eb98 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:09.981667 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 86/173] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jun 21 18:05:09 2022] GET /identity/v3/users/0c89ac265bd34c75a692d91d672fad59 => generated 247 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:09.987670 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c85cb4-f06c-4558-ab4a-4280a036eb98 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:10.032362 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3127657-3c05-4e57-87ca-7b5d8d5b9064 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:10.034499 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:10.123895 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d76138-feec-4a30-ae07-753dc2d3df9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:10.125171 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d76138-feec-4a30-ae07-753dc2d3df9f None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:10.125664 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d76138-feec-4a30-ae07-753dc2d3df9f None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:10.126139 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d76138-feec-4a30-ae07-753dc2d3df9f None admin] PATH_INFO: `/v3/users/623a005b923a4f81bdcab9453639598a` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:10.133319 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d76138-feec-4a30-ae07-753dc2d3df9f None admin] RBAC: Authorizing `identity:get_user(user_id=623a005b923a4f81bdcab9453639598a)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:10.135718 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d76138-feec-4a30-ae07-753dc2d3df9f None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:10.137694 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 88/174] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jun 21 18:05:09 2022] GET /identity/v3/users/623a005b923a4f81bdcab9453639598a => generated 275 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:10.146659 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d76138-feec-4a30-ae07-753dc2d3df9f None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:10.172959 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:10.179153 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c85cb4-f06c-4558-ab4a-4280a036eb98 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:10.259360 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a8d63e-133b-449a-8e47-aa03d7f1c528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:10.260066 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a8d63e-133b-449a-8e47-aa03d7f1c528 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:10.260354 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a8d63e-133b-449a-8e47-aa03d7f1c528 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:10.260658 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a8d63e-133b-449a-8e47-aa03d7f1c528 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:10.368599 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d76138-feec-4a30-ae07-753dc2d3df9f None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:10.406912 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a8d63e-133b-449a-8e47-aa03d7f1c528 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:10.412537 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a8d63e-133b-449a-8e47-aa03d7f1c528 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:10.436572 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4805a532-483d-4c93-b587-a11ed54d39fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:10.437246 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4805a532-483d-4c93-b587-a11ed54d39fe None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:10.437535 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4805a532-483d-4c93-b587-a11ed54d39fe None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:10.437816 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4805a532-483d-4c93-b587-a11ed54d39fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:10.500808 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:10.544408 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:10.559186 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 87/175] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:05:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 21 18:05:10.565500 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a8d63e-133b-449a-8e47-aa03d7f1c528 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:10.569233 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4805a532-483d-4c93-b587-a11ed54d39fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:10.570727 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4805a532-483d-4c93-b587-a11ed54d39fe None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:10.756557 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a8d63e-133b-449a-8e47-aa03d7f1c528 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:10.780613 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 89/176] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:05:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 634 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 21 18:05:10.797489 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4805a532-483d-4c93-b587-a11ed54d39fe None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:10.820295 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f9228d-12be-4778-8830-b711ea369d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:10.821239 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f9228d-12be-4778-8830-b711ea369d02 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:10.821528 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f9228d-12be-4778-8830-b711ea369d02 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:10.821806 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f9228d-12be-4778-8830-b711ea369d02 None admin] PATH_INFO: `/v3/projects/21cd4117dcd346b8bcb3e353ff8d228a` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:10.824460 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f9228d-12be-4778-8830-b711ea369d02 None admin] RBAC: Authorizing `identity:get_project(project_id=21cd4117dcd346b8bcb3e353ff8d228a)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:10.826614 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f9228d-12be-4778-8830-b711ea369d02 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:10.832432 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 88/177] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jun 21 18:05:10 2022] GET /identity/v3/projects/21cd4117dcd346b8bcb3e353ff8d228a => generated 298 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:05:10.848413 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f9228d-12be-4778-8830-b711ea369d02 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:10.933178 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:10.971230 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4805a532-483d-4c93-b587-a11ed54d39fe None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:11.013537 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:11.032433 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e86a97-f5ba-4e49-b967-760a7bb5afa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:11.034357 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e86a97-f5ba-4e49-b967-760a7bb5afa1 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:11.034357 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e86a97-f5ba-4e49-b967-760a7bb5afa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:11.034357 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e86a97-f5ba-4e49-b967-760a7bb5afa1 None admin] PATH_INFO: `/v3/projects/21cd4117dcd346b8bcb3e353ff8d228a` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:11.035891 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e86a97-f5ba-4e49-b967-760a7bb5afa1 None admin] RBAC: Authorizing `identity:get_project(project_id=21cd4117dcd346b8bcb3e353ff8d228a)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:11.037394 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e86a97-f5ba-4e49-b967-760a7bb5afa1 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:11.039308 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 90/178] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jun 21 18:05:10 2022] GET /identity/v3/projects/21cd4117dcd346b8bcb3e353ff8d228a => generated 298 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:11.048560 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54debce4-ad37-4897-89e7-18f05e94438d None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:11.048887 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54debce4-ad37-4897-89e7-18f05e94438d None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:11.049130 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54debce4-ad37-4897-89e7-18f05e94438d None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:11.050714 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 91/179] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:11 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:05:11.053549 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54debce4-ad37-4897-89e7-18f05e94438d None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:11.094807 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f9228d-12be-4778-8830-b711ea369d02 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:11.167480 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8916620d-47d3-44b2-b0f2-f5f69044722f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:11.168564 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8916620d-47d3-44b2-b0f2-f5f69044722f None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:11.168850 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8916620d-47d3-44b2-b0f2-f5f69044722f None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:11.169126 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8916620d-47d3-44b2-b0f2-f5f69044722f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:11.169986 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8916620d-47d3-44b2-b0f2-f5f69044722f None admin] RBAC: Adding query filter params (role.id=745ed9065a8143e384ebbd694b29635c, scope.project.id=21cd4117dcd346b8bcb3e353ff8d228a, user.id=0c89ac265bd34c75a692d91d672fad59) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:11.170299 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8916620d-47d3-44b2-b0f2-f5f69044722f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:11.171784 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8916620d-47d3-44b2-b0f2-f5f69044722f None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:11.183848 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 89/180] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jun 21 18:05:10 2022] GET /identity/v3/role_assignments?user.id=0c89ac265bd34c75a692d91d672fad59&scope.project.id=21cd4117dcd346b8bcb3e353ff8d228a&role.id=745ed9065a8143e384ebbd694b29635c => generated 608 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:11.189398 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dc0cb7-770e-4359-8d32-31623334fa8e None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:11.189689 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dc0cb7-770e-4359-8d32-31623334fa8e None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:11.192936 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dc0cb7-770e-4359-8d32-31623334fa8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:11.223443 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-c1dc0cb7-770e-4359-8d32-31623334fa8e None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:11.223443 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-c1dc0cb7-770e-4359-8d32-31623334fa8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:11.299254 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54debce4-ad37-4897-89e7-18f05e94438d None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:11.376548 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d48a8e6-73b0-4ad1-bbb8-cc97f87ecf35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:11.377228 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d48a8e6-73b0-4ad1-bbb8-cc97f87ecf35 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:11.377519 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d48a8e6-73b0-4ad1-bbb8-cc97f87ecf35 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:11.377803 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d48a8e6-73b0-4ad1-bbb8-cc97f87ecf35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:11.378681 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d48a8e6-73b0-4ad1-bbb8-cc97f87ecf35 None admin] RBAC: Adding query filter params (role.id=482dcc9ba0fd400cad2d193e3e895d4c, scope.project.id=21cd4117dcd346b8bcb3e353ff8d228a, user.id=623a005b923a4f81bdcab9453639598a) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:11.378997 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d48a8e6-73b0-4ad1-bbb8-cc97f87ecf35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:11.380526 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d48a8e6-73b0-4ad1-bbb8-cc97f87ecf35 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:11.390101 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 90/181] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:05:11.393718 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 92/182] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jun 21 18:05:11 2022] GET /identity/v3/role_assignments?user.id=623a005b923a4f81bdcab9453639598a&scope.project.id=21cd4117dcd346b8bcb3e353ff8d228a&role.id=482dcc9ba0fd400cad2d193e3e895d4c => generated 608 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:11.438544 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dc0cb7-770e-4359-8d32-31623334fa8e None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:11.525396 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:11.585562 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dc0cb7-770e-4359-8d32-31623334fa8e None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:11.638715 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d910d6-dde4-42fc-9b74-3eec36738430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:11.639542 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d910d6-dde4-42fc-9b74-3eec36738430 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:11.639828 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d910d6-dde4-42fc-9b74-3eec36738430 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:11.640103 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d910d6-dde4-42fc-9b74-3eec36738430 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:11.654707 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d910d6-dde4-42fc-9b74-3eec36738430 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:05:11.662612 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-80d910d6-dde4-42fc-9b74-3eec36738430 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 18:05:11.662612 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:11.662612 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:11.662612 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:11.662612 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:11.662612 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:11.662612 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:11.662612 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:11.662612 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:11.662612 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:11.662612 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:11.662612 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:11.662612 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:05:11.662612 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:05:11.662612 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:05:11.662612 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:05:11.664262 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:05:11.664262 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:05:11.664262 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:05:11.664262 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:05:11.664262 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:11.664262 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:11.664262 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:11.664262 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:11.664262 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:11.664262 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:11.664262 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:11.664262 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:11.664262 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:11.664262 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:11.664262 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:11.664262 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:11.665860 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:11.665860 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:11.665860 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:11.665860 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:11.665860 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:05:11.665860 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:05:11.665860 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:05:11.665860 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:05:11.665860 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:05:11.665860 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:05:11.665860 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 18:05:11.665860 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:05:11.665860 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 91/183] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:05:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:11.674381 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d48a8e6-73b0-4ad1-bbb8-cc97f87ecf35 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:11.688766 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:11.770837 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b43428-11e0-444b-9514-d797bf5035ce None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:11.770837 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b43428-11e0-444b-9514-d797bf5035ce None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:11.770837 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b43428-11e0-444b-9514-d797bf5035ce None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:11.772217 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 92/184] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:05:11.778476 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf52e29-e786-410a-944f-f34a455247d1 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:11.778764 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf52e29-e786-410a-944f-f34a455247d1 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:11.779045 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf52e29-e786-410a-944f-f34a455247d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:11.807101 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-bcf52e29-e786-410a-944f-f34a455247d1 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:11.808871 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-bcf52e29-e786-410a-944f-f34a455247d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:11.819317 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d48a8e6-73b0-4ad1-bbb8-cc97f87ecf35 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:11.890044 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc153f8-b618-4791-9a43-2bf0038c17c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:11.891360 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc153f8-b618-4791-9a43-2bf0038c17c2 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:11.892001 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc153f8-b618-4791-9a43-2bf0038c17c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:11.892838 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc153f8-b618-4791-9a43-2bf0038c17c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:11.894582 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc153f8-b618-4791-9a43-2bf0038c17c2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:11.895375 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc153f8-b618-4791-9a43-2bf0038c17c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:11.898601 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc153f8-b618-4791-9a43-2bf0038c17c2 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:11.913133 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 93/185] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:05:11.917289 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 93/186] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jun 21 18:05:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:11.922968 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf52e29-e786-410a-944f-f34a455247d1 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:11.943245 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc153f8-b618-4791-9a43-2bf0038c17c2 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:11.977484 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:12.012813 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:12.055208 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc153f8-b618-4791-9a43-2bf0038c17c2 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:12.059459 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf52e29-e786-410a-944f-f34a455247d1 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:12.097949 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f5b288-94ce-4cfd-a5ff-c3a6af24de98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:12.098878 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f5b288-94ce-4cfd-a5ff-c3a6af24de98 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:12.099231 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f5b288-94ce-4cfd-a5ff-c3a6af24de98 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:12.099620 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f5b288-94ce-4cfd-a5ff-c3a6af24de98 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:12.107866 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b070cdb-0c44-4a20-999b-3a08740e7455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:12.109086 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b070cdb-0c44-4a20-999b-3a08740e7455 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:12.109563 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b070cdb-0c44-4a20-999b-3a08740e7455 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:12.109888 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b070cdb-0c44-4a20-999b-3a08740e7455 None admin] PATH_INFO: `/v3/users/40c2a6f1a2794b24b9a92a919a888051` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:12.111091 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b070cdb-0c44-4a20-999b-3a08740e7455 None admin] RBAC: Authorizing `identity:get_user(user_id=40c2a6f1a2794b24b9a92a919a888051)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:12.112016 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f5b288-94ce-4cfd-a5ff-c3a6af24de98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:05:12.112584 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b070cdb-0c44-4a20-999b-3a08740e7455 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:12.114396 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 94/187] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jun 21 18:05:11 2022] GET /identity/v3/users/40c2a6f1a2794b24b9a92a919a888051 => generated 289 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:12.119902 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1e85c-edb7-4701-86da-17adaa9a21e7 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:12.119902 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1e85c-edb7-4701-86da-17adaa9a21e7 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:12.119902 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1e85c-edb7-4701-86da-17adaa9a21e7 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:12.119902 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 95/188] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:05:12.120907 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a1e85c-edb7-4701-86da-17adaa9a21e7 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:12.126113 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-21f5b288-94ce-4cfd-a5ff-c3a6af24de98 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:05:12.126113 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:12.126113 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:12.126113 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:12.126113 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:12.126113 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:12.126113 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:12.126113 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:12.126113 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:12.126113 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:12.126113 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:12.126113 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:12.126113 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:05:12.126113 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:05:12.126113 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:05:12.126113 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:05:12.127040 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:05:12.127040 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:05:12.127040 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:05:12.127040 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:05:12.127040 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:12.127040 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:12.127040 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:12.127040 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:12.127040 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:12.127040 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:12.127040 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:12.127040 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:12.127040 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:12.127040 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:12.127040 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:12.127040 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:12.127872 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:12.127872 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:12.127872 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:12.127872 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:12.127872 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:05:12.127872 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:05:12.127872 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:05:12.127872 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:05:12.127872 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:05:12.127872 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:05:12.127872 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:05:12.127872 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:05:12.127872 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 94/189] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jun 21 18:05:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:12.128595 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc7d16-3dc1-4292-8589-8de03c83a528 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:12.133571 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc7d16-3dc1-4292-8589-8de03c83a528 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:12.133571 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc7d16-3dc1-4292-8589-8de03c83a528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:12.149761 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-2dfc7d16-3dc1-4292-8589-8de03c83a528 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:12.151694 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-2dfc7d16-3dc1-4292-8589-8de03c83a528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:12.226092 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 95/190] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:05:12.228128 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfc7d16-3dc1-4292-8589-8de03c83a528 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:12.244196 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a1e85c-edb7-4701-86da-17adaa9a21e7 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:12.259430 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:12.285651 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f42714-2c43-4980-ac6b-74e9ee4e9de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:12.286308 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f42714-2c43-4980-ac6b-74e9ee4e9de4 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:12.286595 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f42714-2c43-4980-ac6b-74e9ee4e9de4 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:12.286896 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f42714-2c43-4980-ac6b-74e9ee4e9de4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:12.289998 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:12.336974 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfc7d16-3dc1-4292-8589-8de03c83a528 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:12.357346 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f42714-2c43-4980-ac6b-74e9ee4e9de4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:12.358836 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f42714-2c43-4980-ac6b-74e9ee4e9de4 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:12.372376 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451dbb05-49bf-44dd-b25e-c2716079ed30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:12.375472 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451dbb05-49bf-44dd-b25e-c2716079ed30 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:12.375472 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451dbb05-49bf-44dd-b25e-c2716079ed30 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:12.375472 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451dbb05-49bf-44dd-b25e-c2716079ed30 None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:12.375472 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451dbb05-49bf-44dd-b25e-c2716079ed30 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:12.375472 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451dbb05-49bf-44dd-b25e-c2716079ed30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:12.376015 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451dbb05-49bf-44dd-b25e-c2716079ed30 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:12.385333 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 96/191] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:05:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:12.386865 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad855c34-6951-4f03-926f-03f97b11b83a None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:12.387153 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad855c34-6951-4f03-926f-03f97b11b83a None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:12.387428 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad855c34-6951-4f03-926f-03f97b11b83a None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:12.392697 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 97/192] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:05:12.392697 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad855c34-6951-4f03-926f-03f97b11b83a None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:12.467589 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 96/193] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:05:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:05:12.467589 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f42714-2c43-4980-ac6b-74e9ee4e9de4 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:12.483699 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:12.500793 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad855c34-6951-4f03-926f-03f97b11b83a None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:12.517223 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:12.536679 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f214ef39-0a0b-410a-8660-f8938bc2ab58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:12.537479 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f214ef39-0a0b-410a-8660-f8938bc2ab58 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:12.537764 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f214ef39-0a0b-410a-8660-f8938bc2ab58 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:12.538068 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f214ef39-0a0b-410a-8660-f8938bc2ab58 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:12.539230 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f214ef39-0a0b-410a-8660-f8938bc2ab58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:12.540772 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f214ef39-0a0b-410a-8660-f8938bc2ab58 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:12.543074 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 98/194] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:05:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:12.544267 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4905b080-9b4e-458b-bca3-8c97168f55e8 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:12.544587 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4905b080-9b4e-458b-bca3-8c97168f55e8 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:12.544868 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4905b080-9b4e-458b-bca3-8c97168f55e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:12.561644 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-4905b080-9b4e-458b-bca3-8c97168f55e8 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:12.562905 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-4905b080-9b4e-458b-bca3-8c97168f55e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:12.579568 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f42714-2c43-4980-ac6b-74e9ee4e9de4 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:12.616442 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0445167-3674-452d-8707-e52bd7199602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:12.617329 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0445167-3674-452d-8707-e52bd7199602 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:12.617618 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0445167-3674-452d-8707-e52bd7199602 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:12.617892 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0445167-3674-452d-8707-e52bd7199602 None admin] PATH_INFO: `/v3/users/376aeb7be69e4a7bb27f6fbb7772ba41` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:12.631559 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0445167-3674-452d-8707-e52bd7199602 None admin] RBAC: Authorizing `identity:get_user(user_id=376aeb7be69e4a7bb27f6fbb7772ba41)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:12.633035 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 99/195] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:05:12.633142 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0445167-3674-452d-8707-e52bd7199602 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:12.633897 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4905b080-9b4e-458b-bca3-8c97168f55e8 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:12.634949 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 97/196] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jun 21 18:05:12 2022] GET /identity/v3/users/376aeb7be69e4a7bb27f6fbb7772ba41 => generated 283 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:12.635657 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0445167-3674-452d-8707-e52bd7199602 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:12.704794 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:12.739999 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0445167-3674-452d-8707-e52bd7199602 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:12.749597 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4905b080-9b4e-458b-bca3-8c97168f55e8 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:12.764119 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:12.773397 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3b8605-d2b8-4dd6-8921-47c54f90ff15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:12.774087 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b8605-d2b8-4dd6-8921-47c54f90ff15 None admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:12.781931 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b8605-d2b8-4dd6-8921-47c54f90ff15 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:12.781931 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b8605-d2b8-4dd6-8921-47c54f90ff15 None admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:12.781931 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3b8605-d2b8-4dd6-8921-47c54f90ff15 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:12.781931 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3b8605-d2b8-4dd6-8921-47c54f90ff15 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:12.784296 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0eec1b-4584-401f-b9af-a14e35700fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:12.785198 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0eec1b-4584-401f-b9af-a14e35700fde None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:12.785487 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0eec1b-4584-401f-b9af-a14e35700fde None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:12.785767 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0eec1b-4584-401f-b9af-a14e35700fde None admin] PATH_INFO: `/v3/projects/21cd4117dcd346b8bcb3e353ff8d228a` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:12.787025 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0eec1b-4584-401f-b9af-a14e35700fde None admin] RBAC: Authorizing `identity:get_project(project_id=21cd4117dcd346b8bcb3e353ff8d228a)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:12.788511 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0eec1b-4584-401f-b9af-a14e35700fde None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:12.790246 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 100/197] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jun 21 18:05:12 2022] GET /identity/v3/projects/21cd4117dcd346b8bcb3e353ff8d228a => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:12.791048 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0eec1b-4584-401f-b9af-a14e35700fde None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:12.873808 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 98/198] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Jun 21 18:05:12 2022] POST /identity/v3/users => generated 297 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:05:12.874705 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3b8605-d2b8-4dd6-8921-47c54f90ff15 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:12.907939 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0eec1b-4584-401f-b9af-a14e35700fde None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:12.949794 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189f9b6c-04dd-44dd-9bbc-114677717ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:12.950476 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f9b6c-04dd-44dd-9bbc-114677717ced None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:12.950764 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f9b6c-04dd-44dd-9bbc-114677717ced None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:12.951046 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f9b6c-04dd-44dd-9bbc-114677717ced None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:12.999510 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3b8605-d2b8-4dd6-8921-47c54f90ff15 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:13.008009 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:13.028666 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:13.031576 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189f9b6c-04dd-44dd-9bbc-114677717ced None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:13.034443 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189f9b6c-04dd-44dd-9bbc-114677717ced None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:13.045403 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df44052-ed3f-485f-b5d5-a277642d548a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:13.046244 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df44052-ed3f-485f-b5d5-a277642d548a None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:13.047842 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df44052-ed3f-485f-b5d5-a277642d548a None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:13.047842 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df44052-ed3f-485f-b5d5-a277642d548a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:13.060142 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df44052-ed3f-485f-b5d5-a277642d548a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:05:13.070296 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-5df44052-ed3f-485f-b5d5-a277642d548a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:05:13.070296 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:13.070296 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:13.070296 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:13.070296 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:13.070296 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:13.070296 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:13.070296 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:13.070296 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:13.070296 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:13.070296 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:13.070296 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:13.070296 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:05:13.070296 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:05:13.070296 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:05:13.070296 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:05:13.071263 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:05:13.071263 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:05:13.071263 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:05:13.071263 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:05:13.071263 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:13.071263 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:13.071263 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:13.071263 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:13.071263 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:13.071263 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:13.071263 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:13.071263 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:13.071263 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:13.071263 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:13.071263 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:13.071263 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:13.072098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:13.072098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:13.072098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:13.072098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:13.072098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:05:13.072098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:05:13.072098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:05:13.072098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:05:13.072098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:05:13.072098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:05:13.072098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:05:13.072098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:05:13.072098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 99/199] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jun 21 18:05:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:13.073810 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df44052-ed3f-485f-b5d5-a277642d548a None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:13.179378 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 101/200] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:05:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:05:13.180662 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189f9b6c-04dd-44dd-9bbc-114677717ced None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:13.216478 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df44052-ed3f-485f-b5d5-a277642d548a None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:13.271396 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1879b8-5a53-47a6-b228-c026c36854ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:13.273045 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1879b8-5a53-47a6-b228-c026c36854ee None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:13.273810 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1879b8-5a53-47a6-b228-c026c36854ee None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:13.274294 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1879b8-5a53-47a6-b228-c026c36854ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:13.277336 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1879b8-5a53-47a6-b228-c026c36854ee None admin] RBAC: Adding query filter params (role.id=55af05e4b36e44d282933b6a18958580, scope.project.id=21cd4117dcd346b8bcb3e353ff8d228a, user.id=40c2a6f1a2794b24b9a92a919a888051) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:13.277924 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1879b8-5a53-47a6-b228-c026c36854ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:13.279882 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1879b8-5a53-47a6-b228-c026c36854ee None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:13.289797 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:13.290313 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 100/201] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jun 21 18:05:13 2022] GET /identity/v3/role_assignments?user.id=40c2a6f1a2794b24b9a92a919a888051&scope.project.id=21cd4117dcd346b8bcb3e353ff8d228a&role.id=55af05e4b36e44d282933b6a18958580 => generated 608 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:13.293299 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1879b8-5a53-47a6-b228-c026c36854ee None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:13.312362 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189f9b6c-04dd-44dd-9bbc-114677717ced None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:13.352176 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80160877-42be-4f44-9778-b50399f0b591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:13.352885 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80160877-42be-4f44-9778-b50399f0b591 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:13.353168 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80160877-42be-4f44-9778-b50399f0b591 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:13.353442 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80160877-42be-4f44-9778-b50399f0b591 None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:13.354316 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80160877-42be-4f44-9778-b50399f0b591 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:13.354663 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80160877-42be-4f44-9778-b50399f0b591 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:13.356115 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80160877-42be-4f44-9778-b50399f0b591 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:13.365856 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:13.367459 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 102/202] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jun 21 18:05:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:13.374260 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80160877-42be-4f44-9778-b50399f0b591 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:13.438561 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1879b8-5a53-47a6-b228-c026c36854ee None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:13.476952 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f3586f-5797-4f52-97c1-bf86afeaa91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:13.477918 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3586f-5797-4f52-97c1-bf86afeaa91c None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:13.478199 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3586f-5797-4f52-97c1-bf86afeaa91c None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:13.478505 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3586f-5797-4f52-97c1-bf86afeaa91c None admin] PATH_INFO: `/v3/projects/53e988a4b11648fcb59e0748d5dbf37b` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:13.480137 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f3586f-5797-4f52-97c1-bf86afeaa91c None admin] RBAC: Authorizing `identity:get_project(project_id=53e988a4b11648fcb59e0748d5dbf37b)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:13.481529 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f3586f-5797-4f52-97c1-bf86afeaa91c None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:13.483670 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 101/203] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jun 21 18:05:13 2022] GET /identity/v3/projects/53e988a4b11648fcb59e0748d5dbf37b => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:13.489735 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f3586f-5797-4f52-97c1-bf86afeaa91c None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:13.510870 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80160877-42be-4f44-9778-b50399f0b591 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:13.564345 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12aee19-c658-4a38-aefb-fc3cb26fe984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:13.566870 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12aee19-c658-4a38-aefb-fc3cb26fe984 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:13.567160 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12aee19-c658-4a38-aefb-fc3cb26fe984 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:13.567447 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12aee19-c658-4a38-aefb-fc3cb26fe984 None admin] PATH_INFO: `/v3/users/376aeb7be69e4a7bb27f6fbb7772ba41` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:13.570544 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12aee19-c658-4a38-aefb-fc3cb26fe984 None admin] RBAC: Authorizing `identity:get_user(user_id=376aeb7be69e4a7bb27f6fbb7772ba41)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:13.572060 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12aee19-c658-4a38-aefb-fc3cb26fe984 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:13.574455 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 103/204] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jun 21 18:05:13 2022] GET /identity/v3/users/376aeb7be69e4a7bb27f6fbb7772ba41 => generated 283 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:13.585934 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12aee19-c658-4a38-aefb-fc3cb26fe984 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:13.609217 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:13.619395 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f3586f-5797-4f52-97c1-bf86afeaa91c None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:13.646045 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:13.672881 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e6b2e5-cfef-4d6b-ac28-7275d7981925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:13.673611 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6b2e5-cfef-4d6b-ac28-7275d7981925 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:13.673935 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6b2e5-cfef-4d6b-ac28-7275d7981925 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:13.674182 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6b2e5-cfef-4d6b-ac28-7275d7981925 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:13.675078 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e6b2e5-cfef-4d6b-ac28-7275d7981925 None admin] RBAC: Adding query filter params (role.id=745ed9065a8143e384ebbd694b29635c, scope.project.id=53e988a4b11648fcb59e0748d5dbf37b, user.id=376aeb7be69e4a7bb27f6fbb7772ba41) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:13.675449 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e6b2e5-cfef-4d6b-ac28-7275d7981925 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:13.677187 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e6b2e5-cfef-4d6b-ac28-7275d7981925 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:13.688810 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 102/205] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jun 21 18:05:13 2022] GET /identity/v3/role_assignments?user.id=376aeb7be69e4a7bb27f6fbb7772ba41&scope.project.id=53e988a4b11648fcb59e0748d5dbf37b&role.id=745ed9065a8143e384ebbd694b29635c => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:05:13.729239 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12aee19-c658-4a38-aefb-fc3cb26fe984 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:13.770959 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722cdb12-4b7f-4b02-92b2-e14510199142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:13.772419 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722cdb12-4b7f-4b02-92b2-e14510199142 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:13.772862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722cdb12-4b7f-4b02-92b2-e14510199142 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:13.773313 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722cdb12-4b7f-4b02-92b2-e14510199142 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:13.856600 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722cdb12-4b7f-4b02-92b2-e14510199142 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:13.858301 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722cdb12-4b7f-4b02-92b2-e14510199142 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:13.877964 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:13.967789 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 104/206] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:05:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 21 18:05:13.973272 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e6b2e5-cfef-4d6b-ac28-7275d7981925 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:14.086066 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e6b2e5-cfef-4d6b-ac28-7275d7981925 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:14.127348 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf2cd74-d0ac-4793-96a0-67155025d679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:14.128516 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2cd74-d0ac-4793-96a0-67155025d679 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:14.129175 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2cd74-d0ac-4793-96a0-67155025d679 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:14.129644 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2cd74-d0ac-4793-96a0-67155025d679 None admin] PATH_INFO: `/v3/projects/53e988a4b11648fcb59e0748d5dbf37b` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:14.131016 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf2cd74-d0ac-4793-96a0-67155025d679 None admin] RBAC: Authorizing `identity:get_project(project_id=53e988a4b11648fcb59e0748d5dbf37b)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:14.132743 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf2cd74-d0ac-4793-96a0-67155025d679 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:14.134935 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 103/207] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jun 21 18:05:13 2022] GET /identity/v3/projects/53e988a4b11648fcb59e0748d5dbf37b => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:14.141167 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722cdb12-4b7f-4b02-92b2-e14510199142 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:14.264675 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722cdb12-4b7f-4b02-92b2-e14510199142 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:14.280717 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:14.304554 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ebae75-e08c-4bab-ae2d-9b60a0a3f5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:14.305229 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ebae75-e08c-4bab-ae2d-9b60a0a3f5d1 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:14.305515 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ebae75-e08c-4bab-ae2d-9b60a0a3f5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:14.305796 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ebae75-e08c-4bab-ae2d-9b60a0a3f5d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:14.306643 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ebae75-e08c-4bab-ae2d-9b60a0a3f5d1 None admin] RBAC: Adding query filter params (role.id=482dcc9ba0fd400cad2d193e3e895d4c, scope.project.id=53e988a4b11648fcb59e0748d5dbf37b, user.id=376aeb7be69e4a7bb27f6fbb7772ba41) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:14.307612 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ebae75-e08c-4bab-ae2d-9b60a0a3f5d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:14.309085 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ebae75-e08c-4bab-ae2d-9b60a0a3f5d1 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:14.315328 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 105/208] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jun 21 18:05:14 2022] GET /identity/v3/role_assignments?user.id=376aeb7be69e4a7bb27f6fbb7772ba41&scope.project.id=53e988a4b11648fcb59e0748d5dbf37b&role.id=482dcc9ba0fd400cad2d193e3e895d4c => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:15.214586 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74befd0-fb38-4ef4-b063-fd27794505b6 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:15.215105 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74befd0-fb38-4ef4-b063-fd27794505b6 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:15.215183 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74befd0-fb38-4ef4-b063-fd27794505b6 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:15.216559 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 104/209] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:05:15.221965 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ea319d-f889-4926-bcd2-9dc4e9023a2d None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:15.222656 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ea319d-f889-4926-bcd2-9dc4e9023a2d None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:15.222823 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ea319d-f889-4926-bcd2-9dc4e9023a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:15.239952 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-a4ea319d-f889-4926-bcd2-9dc4e9023a2d None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:15.241195 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-a4ea319d-f889-4926-bcd2-9dc4e9023a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:15.312490 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 106/210] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:05:15.336760 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74befd0-fb38-4ef4-b063-fd27794505b6 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:15.390220 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:15.446262 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74befd0-fb38-4ef4-b063-fd27794505b6 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:15.485164 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0deb4a-2cea-4565-b9f3-dafffa6c0681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:15.485906 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0deb4a-2cea-4565-b9f3-dafffa6c0681 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:15.486189 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0deb4a-2cea-4565-b9f3-dafffa6c0681 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:15.486465 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0deb4a-2cea-4565-b9f3-dafffa6c0681 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:15.487716 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0deb4a-2cea-4565-b9f3-dafffa6c0681 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:15.489224 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0deb4a-2cea-4565-b9f3-dafffa6c0681 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:15.490891 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 105/211] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:05:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:15.497722 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ea319d-f889-4926-bcd2-9dc4e9023a2d None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:15.602396 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ea319d-f889-4926-bcd2-9dc4e9023a2d None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:15.648477 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:15.676900 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29618036-f7e7-4eac-9b3d-b2869052882a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:15.677642 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29618036-f7e7-4eac-9b3d-b2869052882a None admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:15.677929 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29618036-f7e7-4eac-9b3d-b2869052882a None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:15.678207 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29618036-f7e7-4eac-9b3d-b2869052882a None admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:15.679026 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29618036-f7e7-4eac-9b3d-b2869052882a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:15.680778 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29618036-f7e7-4eac-9b3d-b2869052882a None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:15.761515 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04513d-47d2-4654-bd08-bc3aa7508fb6 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:15.761965 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04513d-47d2-4654-bd08-bc3aa7508fb6 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:15.762471 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04513d-47d2-4654-bd08-bc3aa7508fb6 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:15.765147 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 106/212] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:05:15.771404 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d4399c-e26c-46bc-ae25-5bcbcdc594b1 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:15.772065 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d4399c-e26c-46bc-ae25-5bcbcdc594b1 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:15.772722 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d4399c-e26c-46bc-ae25-5bcbcdc594b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:15.788947 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 107/213] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Jun 21 18:05:15 2022] POST /identity/v3/users => generated 297 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:05:15.804918 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-45d4399c-e26c-46bc-ae25-5bcbcdc594b1 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:15.806183 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-45d4399c-e26c-46bc-ae25-5bcbcdc594b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:15.875774 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eac64f-2327-4ffa-bc4d-33b6e286599c None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:15.876123 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eac64f-2327-4ffa-bc4d-33b6e286599c None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:15.876338 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eac64f-2327-4ffa-bc4d-33b6e286599c None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:15.878232 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 108/214] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:05:15.889669 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1098c445-7ff2-4f02-bcd7-a45ce69cd104 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:15.889962 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1098c445-7ff2-4f02-bcd7-a45ce69cd104 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:15.890456 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1098c445-7ff2-4f02-bcd7-a45ce69cd104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:15.898352 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 107/215] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:05:15.910537 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-1098c445-7ff2-4f02-bcd7-a45ce69cd104 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:15.912036 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-1098c445-7ff2-4f02-bcd7-a45ce69cd104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:15.926505 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d4399c-e26c-46bc-ae25-5bcbcdc594b1 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:15.983510 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:16.014230 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 109/216] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:05:16.036930 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1098c445-7ff2-4f02-bcd7-a45ce69cd104 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:16.059179 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d4399c-e26c-46bc-ae25-5bcbcdc594b1 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:16.098021 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357f471f-e2ec-42d4-8929-c193dc705583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:16.099086 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357f471f-e2ec-42d4-8929-c193dc705583 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:16.100242 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357f471f-e2ec-42d4-8929-c193dc705583 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:16.100242 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357f471f-e2ec-42d4-8929-c193dc705583 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:16.114873 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357f471f-e2ec-42d4-8929-c193dc705583 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:05:16.123077 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-357f471f-e2ec-42d4-8929-c193dc705583 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:05:16.123077 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:16.123077 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:16.123077 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:16.123077 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:16.123077 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:16.123077 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:16.123077 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:16.123077 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:16.123077 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:16.123077 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:16.123077 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:16.123077 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:05:16.123077 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:05:16.123077 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:05:16.123077 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:05:16.124043 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:05:16.124043 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:05:16.124043 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:05:16.124043 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:05:16.124043 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:16.124043 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:16.124043 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:16.124043 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:16.124043 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:16.124043 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:16.124043 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:16.124043 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:16.124043 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:16.124043 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:16.124043 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:16.124043 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:16.124908 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:16.124908 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:16.124908 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:16.124908 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:16.124908 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:05:16.124908 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:05:16.124908 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:05:16.124908 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:05:16.124908 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:05:16.124908 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:05:16.124908 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:05:16.124908 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:05:16.124908 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 108/217] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:05:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:16.131162 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357f471f-e2ec-42d4-8929-c193dc705583 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:16.162910 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:16.174948 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1098c445-7ff2-4f02-bcd7-a45ce69cd104 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:16.222034 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be09b577-6811-40fe-8e53-f7ae424d406a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:16.223098 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09b577-6811-40fe-8e53-f7ae424d406a None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:16.223383 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09b577-6811-40fe-8e53-f7ae424d406a None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:16.223761 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09b577-6811-40fe-8e53-f7ae424d406a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:16.246259 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be09b577-6811-40fe-8e53-f7ae424d406a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:05:16.254989 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-be09b577-6811-40fe-8e53-f7ae424d406a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:05:16.254989 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:16.254989 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:16.254989 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:16.254989 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:16.254989 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:16.254989 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:16.254989 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:16.254989 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:16.254989 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:16.254989 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:16.254989 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:16.254989 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:05:16.254989 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:05:16.254989 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:05:16.254989 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:05:16.255976 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:05:16.255976 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:05:16.255976 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:05:16.255976 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:05:16.255976 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:16.255976 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:16.255976 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:16.255976 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:16.255976 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:16.255976 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:16.255976 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:16.255976 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:16.255976 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:16.255976 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:16.255976 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:16.255976 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:16.256822 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:16.256822 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:16.256822 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:16.256822 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:16.256822 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:05:16.256822 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:05:16.256822 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:05:16.256822 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:05:16.256822 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:05:16.256822 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:05:16.256822 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:05:16.256822 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:05:16.256822 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 110/218] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jun 21 18:05:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:05:16.264951 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be09b577-6811-40fe-8e53-f7ae424d406a None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:16.314694 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357f471f-e2ec-42d4-8929-c193dc705583 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:16.327420 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:16.382708 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf67f29-1c07-4ae3-9903-a2999cf2254b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:16.383397 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf67f29-1c07-4ae3-9903-a2999cf2254b None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:16.383688 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf67f29-1c07-4ae3-9903-a2999cf2254b None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:16.383973 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf67f29-1c07-4ae3-9903-a2999cf2254b None admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:16.384772 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf67f29-1c07-4ae3-9903-a2999cf2254b None admin] RBAC: Adding query filter params (name=member) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:16.385107 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf67f29-1c07-4ae3-9903-a2999cf2254b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:16.392194 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf67f29-1c07-4ae3-9903-a2999cf2254b None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:16.410820 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 109/219] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jun 21 18:05:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:16.416017 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf67f29-1c07-4ae3-9903-a2999cf2254b None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:16.433263 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be09b577-6811-40fe-8e53-f7ae424d406a None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:16.471904 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd80942-efcb-44f9-9d93-fc8aaa269ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:16.472910 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd80942-efcb-44f9-9d93-fc8aaa269ea3 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:16.473201 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd80942-efcb-44f9-9d93-fc8aaa269ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:16.473486 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd80942-efcb-44f9-9d93-fc8aaa269ea3 None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:16.474257 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd80942-efcb-44f9-9d93-fc8aaa269ea3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:16.474624 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd80942-efcb-44f9-9d93-fc8aaa269ea3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:16.476019 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd80942-efcb-44f9-9d93-fc8aaa269ea3 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:16.488237 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 111/220] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:05:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:16.497067 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd80942-efcb-44f9-9d93-fc8aaa269ea3 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:16.530080 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:16.626815 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf67f29-1c07-4ae3-9903-a2999cf2254b None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:16.670180 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd80942-efcb-44f9-9d93-fc8aaa269ea3 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:16.677096 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b03835-59c9-4dbb-88ff-39d68799058c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:16.677995 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b03835-59c9-4dbb-88ff-39d68799058c None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:16.678284 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b03835-59c9-4dbb-88ff-39d68799058c None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:16.678568 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b03835-59c9-4dbb-88ff-39d68799058c None admin] PATH_INFO: `/v3/users/3d12af51cf5b4a3eb07c7ef257ff00f6` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:16.700373 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b03835-59c9-4dbb-88ff-39d68799058c None admin] RBAC: Authorizing `identity:get_user(user_id=3d12af51cf5b4a3eb07c7ef257ff00f6)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:16.702208 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b03835-59c9-4dbb-88ff-39d68799058c None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:16.705283 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 110/221] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jun 21 18:05:16 2022] GET /identity/v3/users/3d12af51cf5b4a3eb07c7ef257ff00f6 => generated 297 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:05:16.707429 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c05606b-3f21-438c-b3ac-e24493c0b1ce None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:16.707727 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c05606b-3f21-438c-b3ac-e24493c0b1ce None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:16.708118 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c05606b-3f21-438c-b3ac-e24493c0b1ce None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:16.709918 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 111/222] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:05:16.715508 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c05606b-3f21-438c-b3ac-e24493c0b1ce None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:16.717457 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7944b54-22c8-446b-899a-2821a7ee8fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:16.718417 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7944b54-22c8-446b-899a-2821a7ee8fd2 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:16.718710 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7944b54-22c8-446b-899a-2821a7ee8fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:16.718984 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7944b54-22c8-446b-899a-2821a7ee8fd2 None admin] PATH_INFO: `/v3/users/376aeb7be69e4a7bb27f6fbb7772ba41` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:16.720375 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7944b54-22c8-446b-899a-2821a7ee8fd2 None admin] RBAC: Authorizing `identity:get_user(user_id=376aeb7be69e4a7bb27f6fbb7772ba41)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:16.722114 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7944b54-22c8-446b-899a-2821a7ee8fd2 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:16.724530 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 112/223] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jun 21 18:05:16 2022] GET /identity/v3/users/376aeb7be69e4a7bb27f6fbb7772ba41 => generated 283 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:16.725888 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b96c3d9-3cb4-47b0-943b-98e16e306c3d None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:16.726177 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b96c3d9-3cb4-47b0-943b-98e16e306c3d None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:16.726479 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b96c3d9-3cb4-47b0-943b-98e16e306c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:16.748289 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-4b96c3d9-3cb4-47b0-943b-98e16e306c3d None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:16.749920 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-4b96c3d9-3cb4-47b0-943b-98e16e306c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:16.752688 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:16.836492 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 113/224] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:05:16.836492 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b96c3d9-3cb4-47b0-943b-98e16e306c3d None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:16.863420 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:16.873571 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c05606b-3f21-438c-b3ac-e24493c0b1ce None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:16.926040 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19986af-afff-4607-b4de-46de064e6771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:16.928505 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19986af-afff-4607-b4de-46de064e6771 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:16.929578 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19986af-afff-4607-b4de-46de064e6771 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:16.930372 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19986af-afff-4607-b4de-46de064e6771 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:16.967231 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b96c3d9-3cb4-47b0-943b-98e16e306c3d None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:17.006202 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671fcb18-ed13-4b03-bbcf-55deb1d23ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:17.007078 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671fcb18-ed13-4b03-bbcf-55deb1d23ca7 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:17.007358 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671fcb18-ed13-4b03-bbcf-55deb1d23ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:17.007640 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671fcb18-ed13-4b03-bbcf-55deb1d23ca7 None admin] PATH_INFO: `/v3/projects/53e988a4b11648fcb59e0748d5dbf37b` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:17.008980 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671fcb18-ed13-4b03-bbcf-55deb1d23ca7 None admin] RBAC: Authorizing `identity:get_project(project_id=53e988a4b11648fcb59e0748d5dbf37b)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:17.011369 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671fcb18-ed13-4b03-bbcf-55deb1d23ca7 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:17.013524 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 114/225] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jun 21 18:05:16 2022] GET /identity/v3/projects/53e988a4b11648fcb59e0748d5dbf37b => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:17.014366 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671fcb18-ed13-4b03-bbcf-55deb1d23ca7 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:17.085016 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19986af-afff-4607-b4de-46de064e6771 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:17.088707 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19986af-afff-4607-b4de-46de064e6771 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:17.104162 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:17.125440 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671fcb18-ed13-4b03-bbcf-55deb1d23ca7 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:17.137725 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:17.164761 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c490027-2314-40f4-8f3a-a29850ae4683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:17.165626 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c490027-2314-40f4-8f3a-a29850ae4683 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:17.165935 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c490027-2314-40f4-8f3a-a29850ae4683 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:17.167423 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c490027-2314-40f4-8f3a-a29850ae4683 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:17.179249 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c490027-2314-40f4-8f3a-a29850ae4683 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:05:17.188593 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-5c490027-2314-40f4-8f3a-a29850ae4683 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:05:17.188593 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:17.188593 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:17.188593 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:17.188593 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:17.188593 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:17.188593 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:17.188593 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:17.188593 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:17.188593 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:17.188593 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:17.188593 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:17.188593 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:05:17.188593 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:05:17.188593 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:05:17.188593 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:05:17.189569 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:05:17.189569 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:05:17.189569 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:05:17.189569 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:05:17.189569 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:17.189569 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:17.189569 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:17.189569 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:17.189569 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:17.189569 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:17.189569 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:17.189569 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:17.189569 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:17.189569 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:17.189569 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:17.189569 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:17.190385 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:17.190385 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:17.190385 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:17.190385 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:17.190385 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:05:17.190385 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:05:17.190385 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:05:17.190385 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:05:17.190385 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:05:17.190385 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:05:17.190385 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:05:17.190385 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:05:17.190385 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 115/226] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jun 21 18:05:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:17.191058 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c490027-2314-40f4-8f3a-a29850ae4683 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:17.250018 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 112/227] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:05:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:05:17.251703 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19986af-afff-4607-b4de-46de064e6771 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:17.310344 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c490027-2314-40f4-8f3a-a29850ae4683 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:17.361915 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dce25a-d0be-4881-ad3f-0ba5211af301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:17.362931 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dce25a-d0be-4881-ad3f-0ba5211af301 None admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:17.363177 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dce25a-d0be-4881-ad3f-0ba5211af301 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:17.363456 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dce25a-d0be-4881-ad3f-0ba5211af301 None admin] PATH_INFO: `/v3/projects/53e988a4b11648fcb59e0748d5dbf37b/users/376aeb7be69e4a7bb27f6fbb7772ba41/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:17.373441 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dce25a-d0be-4881-ad3f-0ba5211af301 None admin] RBAC: Authorizing `identity:create_grant(project_id=53e988a4b11648fcb59e0748d5dbf37b, user_id=376aeb7be69e4a7bb27f6fbb7772ba41, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:17.374993 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dce25a-d0be-4881-ad3f-0ba5211af301 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:17.440538 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:17.442860 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19986af-afff-4607-b4de-46de064e6771 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:17.452422 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 116/228] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jun 21 18:05:17 2022] PUT /identity/v3/projects/53e988a4b11648fcb59e0748d5dbf37b/users/376aeb7be69e4a7bb27f6fbb7772ba41/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:05:17.452422 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dce25a-d0be-4881-ad3f-0ba5211af301 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:17.542204 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:17.559909 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c42a3c-4c26-4906-b687-c78a31d3eeb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:17.561306 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c42a3c-4c26-4906-b687-c78a31d3eeb7 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:17.562439 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c42a3c-4c26-4906-b687-c78a31d3eeb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:17.563006 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c42a3c-4c26-4906-b687-c78a31d3eeb7 None admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:17.565171 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c42a3c-4c26-4906-b687-c78a31d3eeb7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:17.565872 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c42a3c-4c26-4906-b687-c78a31d3eeb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:17.569698 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c42a3c-4c26-4906-b687-c78a31d3eeb7 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:17.582070 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 113/229] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jun 21 18:05:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:17.588231 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c42a3c-4c26-4906-b687-c78a31d3eeb7 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:17.756338 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dce25a-d0be-4881-ad3f-0ba5211af301 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:17.805824 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75ca30f-a311-45aa-a0de-722efa6ac1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:17.807232 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ca30f-a311-45aa-a0de-722efa6ac1e8 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:17.807598 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ca30f-a311-45aa-a0de-722efa6ac1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:17.807879 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ca30f-a311-45aa-a0de-722efa6ac1e8 None admin] PATH_INFO: `/v3/projects/53e988a4b11648fcb59e0748d5dbf37b` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:17.809349 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75ca30f-a311-45aa-a0de-722efa6ac1e8 None admin] RBAC: Authorizing `identity:get_project(project_id=53e988a4b11648fcb59e0748d5dbf37b)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:17.810867 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75ca30f-a311-45aa-a0de-722efa6ac1e8 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:17.816160 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 117/230] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jun 21 18:05:17 2022] GET /identity/v3/projects/53e988a4b11648fcb59e0748d5dbf37b => generated 302 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:17.825593 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75ca30f-a311-45aa-a0de-722efa6ac1e8 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:17.889064 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c42a3c-4c26-4906-b687-c78a31d3eeb7 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:17.939931 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:17.980346 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff082a37-5f1b-41d4-8c62-07e3cf7b729f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:17.981311 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff082a37-5f1b-41d4-8c62-07e3cf7b729f None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:17.981642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff082a37-5f1b-41d4-8c62-07e3cf7b729f None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:17.981926 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff082a37-5f1b-41d4-8c62-07e3cf7b729f None admin] PATH_INFO: `/v3/users/376aeb7be69e4a7bb27f6fbb7772ba41` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:17.984426 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff082a37-5f1b-41d4-8c62-07e3cf7b729f None admin] RBAC: Authorizing `identity:get_user(user_id=376aeb7be69e4a7bb27f6fbb7772ba41)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:17.985997 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff082a37-5f1b-41d4-8c62-07e3cf7b729f None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:17.987718 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 114/231] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jun 21 18:05:17 2022] GET /identity/v3/users/376aeb7be69e4a7bb27f6fbb7772ba41 => generated 283 bytes in 400 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:17.995444 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff082a37-5f1b-41d4-8c62-07e3cf7b729f None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:18.027548 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75ca30f-a311-45aa-a0de-722efa6ac1e8 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:18.123649 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13de517c-e02d-49a7-ba59-09ebb4f84b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:18.124340 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13de517c-e02d-49a7-ba59-09ebb4f84b8d None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:18.124786 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13de517c-e02d-49a7-ba59-09ebb4f84b8d None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:18.125074 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13de517c-e02d-49a7-ba59-09ebb4f84b8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:18.125929 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13de517c-e02d-49a7-ba59-09ebb4f84b8d None admin] RBAC: Adding query filter params (role.id=c467f10d3dba4f2dbfd42068f42f74b6, scope.project.id=53e988a4b11648fcb59e0748d5dbf37b, user.id=3d12af51cf5b4a3eb07c7ef257ff00f6) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:18.126246 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13de517c-e02d-49a7-ba59-09ebb4f84b8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:18.127697 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13de517c-e02d-49a7-ba59-09ebb4f84b8d None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:18.144690 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 118/232] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jun 21 18:05:17 2022] GET /identity/v3/role_assignments?user.id=3d12af51cf5b4a3eb07c7ef257ff00f6&scope.project.id=53e988a4b11648fcb59e0748d5dbf37b&role.id=c467f10d3dba4f2dbfd42068f42f74b6 => generated 259 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:18.148690 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:18.245944 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff082a37-5f1b-41d4-8c62-07e3cf7b729f None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:18.266887 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d3e969-4f31-4235-9175-0400162b776a None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:18.267237 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d3e969-4f31-4235-9175-0400162b776a None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:18.267447 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d3e969-4f31-4235-9175-0400162b776a None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:18.269321 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 119/233] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:05:18.275138 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9e418d-c17b-429d-b69b-9e002c24d83a None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:18.275456 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9e418d-c17b-429d-b69b-9e002c24d83a None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:18.275741 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9e418d-c17b-429d-b69b-9e002c24d83a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:18.295669 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-9b9e418d-c17b-429d-b69b-9e002c24d83a None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:18.296834 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-9b9e418d-c17b-429d-b69b-9e002c24d83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:18.318181 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490f9226-5fdd-4273-b6a0-886a58414628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:18.319450 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f9226-5fdd-4273-b6a0-886a58414628 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:18.319955 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f9226-5fdd-4273-b6a0-886a58414628 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:18.320499 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f9226-5fdd-4273-b6a0-886a58414628 None admin] PATH_INFO: `/v3/projects/53e988a4b11648fcb59e0748d5dbf37b` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:18.321922 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490f9226-5fdd-4273-b6a0-886a58414628 None admin] RBAC: Authorizing `identity:get_project(project_id=53e988a4b11648fcb59e0748d5dbf37b)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:18.324076 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490f9226-5fdd-4273-b6a0-886a58414628 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:18.326436 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 115/234] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jun 21 18:05:17 2022] GET /identity/v3/projects/53e988a4b11648fcb59e0748d5dbf37b => generated 302 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:18.332307 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490f9226-5fdd-4273-b6a0-886a58414628 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:18.380574 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 120/235] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:05:18.407195 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9e418d-c17b-429d-b69b-9e002c24d83a None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:18.478039 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:18.488642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490f9226-5fdd-4273-b6a0-886a58414628 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:18.520673 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:18.531078 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9e418d-c17b-429d-b69b-9e002c24d83a None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:18.535958 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466808fe-85fc-4392-9b69-52e6a0bb32c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:18.536978 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466808fe-85fc-4392-9b69-52e6a0bb32c2 None admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:18.537261 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466808fe-85fc-4392-9b69-52e6a0bb32c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:18.537539 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466808fe-85fc-4392-9b69-52e6a0bb32c2 None admin] PATH_INFO: `/v3/projects/53e988a4b11648fcb59e0748d5dbf37b/users/376aeb7be69e4a7bb27f6fbb7772ba41/roles/482dcc9ba0fd400cad2d193e3e895d4c` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:18.543820 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466808fe-85fc-4392-9b69-52e6a0bb32c2 None admin] RBAC: Authorizing `identity:create_grant(project_id=53e988a4b11648fcb59e0748d5dbf37b, user_id=376aeb7be69e4a7bb27f6fbb7772ba41, role_id=482dcc9ba0fd400cad2d193e3e895d4c)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:18.543820 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466808fe-85fc-4392-9b69-52e6a0bb32c2 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:18.583780 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a67f6c4-d499-40de-b49f-cd7053e73d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:18.583780 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a67f6c4-d499-40de-b49f-cd7053e73d79 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:18.583780 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a67f6c4-d499-40de-b49f-cd7053e73d79 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:18.583780 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a67f6c4-d499-40de-b49f-cd7053e73d79 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:18.584189 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a67f6c4-d499-40de-b49f-cd7053e73d79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:18.584189 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a67f6c4-d499-40de-b49f-cd7053e73d79 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:18.584189 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 121/236] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:05:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:05:18.584189 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b77da1-9468-4609-887b-6cccb72500d9 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:18.584189 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b77da1-9468-4609-887b-6cccb72500d9 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:18.584189 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b77da1-9468-4609-887b-6cccb72500d9 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:18.585243 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 122/237] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:05:18.586506 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b77da1-9468-4609-887b-6cccb72500d9 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:18.691846 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 116/238] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jun 21 18:05:18 2022] PUT /identity/v3/projects/53e988a4b11648fcb59e0748d5dbf37b/users/376aeb7be69e4a7bb27f6fbb7772ba41/roles/482dcc9ba0fd400cad2d193e3e895d4c => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:05:18.693012 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ec5b3-1367-449a-bf1e-8c20a5424f93 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:18.693309 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ec5b3-1367-449a-bf1e-8c20a5424f93 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:18.693590 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ec5b3-1367-449a-bf1e-8c20a5424f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:18.711695 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b77da1-9468-4609-887b-6cccb72500d9 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:18.713377 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-2d8ec5b3-1367-449a-bf1e-8c20a5424f93 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:18.714911 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-2d8ec5b3-1367-449a-bf1e-8c20a5424f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:18.757143 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:18.767309 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c9d8b-6c36-4635-afc0-40e2dc479bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:18.768005 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c9d8b-6c36-4635-afc0-40e2dc479bd4 None admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:18.768284 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c9d8b-6c36-4635-afc0-40e2dc479bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:18.768592 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c9d8b-6c36-4635-afc0-40e2dc479bd4 None admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:18.770431 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132c9d8b-6c36-4635-afc0-40e2dc479bd4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:18.772195 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132c9d8b-6c36-4635-afc0-40e2dc479bd4 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:18.828760 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 117/239] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:05:18.855040 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8ec5b3-1367-449a-bf1e-8c20a5424f93 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:18.927105 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 123/240] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Jun 21 18:05:18 2022] POST /identity/v3/users => generated 293 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:05:18.986524 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8ec5b3-1367-449a-bf1e-8c20a5424f93 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:19.010919 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:19.036648 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72850079-61d2-4a53-afc8-bc5b9bfa032d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:19.037525 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72850079-61d2-4a53-afc8-bc5b9bfa032d None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:19.037871 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72850079-61d2-4a53-afc8-bc5b9bfa032d None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:19.038147 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72850079-61d2-4a53-afc8-bc5b9bfa032d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:19.051073 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72850079-61d2-4a53-afc8-bc5b9bfa032d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:05:19.060342 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-72850079-61d2-4a53-afc8-bc5b9bfa032d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 18:05:19.060342 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:19.060342 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:19.060342 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:19.060342 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:19.060342 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:19.060342 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:19.060342 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:19.060342 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:19.060342 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:19.060342 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:19.060342 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:19.060342 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:05:19.060342 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:05:19.060342 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:05:19.060342 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:05:19.061336 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:05:19.061336 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:05:19.061336 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:05:19.061336 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:05:19.061336 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:19.061336 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:19.061336 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:19.061336 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:19.061336 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:19.061336 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:19.061336 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:19.061336 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:19.061336 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:19.061336 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:19.061336 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:19.061336 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:19.062149 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:19.062149 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:19.062149 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:19.062149 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:19.062149 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:05:19.062149 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:05:19.062149 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:05:19.062149 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:05:19.062149 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:05:19.062149 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:05:19.062149 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 18:05:19.062149 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:05:19.062149 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 118/241] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:05:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:05:19.069064 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c9d8b-6c36-4635-afc0-40e2dc479bd4 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:19.219488 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c9d8b-6c36-4635-afc0-40e2dc479bd4 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:19.265180 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffff61e-b34e-4b71-b519-fcfc1db8ae48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:19.265879 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffff61e-b34e-4b71-b519-fcfc1db8ae48 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:19.266166 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffff61e-b34e-4b71-b519-fcfc1db8ae48 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:19.266445 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffff61e-b34e-4b71-b519-fcfc1db8ae48 None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:19.267215 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffff61e-b34e-4b71-b519-fcfc1db8ae48 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:19.267527 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffff61e-b34e-4b71-b519-fcfc1db8ae48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:19.268960 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffff61e-b34e-4b71-b519-fcfc1db8ae48 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:19.278102 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 124/242] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jun 21 18:05:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:19.283643 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72850079-61d2-4a53-afc8-bc5b9bfa032d None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:19.395567 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72850079-61d2-4a53-afc8-bc5b9bfa032d None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:19.437319 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b6cf69-e1b9-413e-a51f-4610e1efcd4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:19.438222 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6cf69-e1b9-413e-a51f-4610e1efcd4e None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:19.438511 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6cf69-e1b9-413e-a51f-4610e1efcd4e None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:19.438793 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6cf69-e1b9-413e-a51f-4610e1efcd4e None admin] PATH_INFO: `/v3/users/33b903b692624b40923cc6fc17ce4c17` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:19.457307 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b6cf69-e1b9-413e-a51f-4610e1efcd4e None admin] RBAC: Authorizing `identity:get_user(user_id=33b903b692624b40923cc6fc17ce4c17)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:19.458813 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b6cf69-e1b9-413e-a51f-4610e1efcd4e None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:19.460879 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 119/243] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jun 21 18:05:19 2022] GET /identity/v3/users/33b903b692624b40923cc6fc17ce4c17 => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:19.466495 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffff61e-b34e-4b71-b519-fcfc1db8ae48 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:19.532219 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:19.595333 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffff61e-b34e-4b71-b519-fcfc1db8ae48 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:19.639808 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6c40be-0dc6-4610-8118-b05e59ee4bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:19.640533 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6c40be-0dc6-4610-8118-b05e59ee4bef None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:19.640825 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6c40be-0dc6-4610-8118-b05e59ee4bef None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:19.641108 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6c40be-0dc6-4610-8118-b05e59ee4bef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:19.727391 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6c40be-0dc6-4610-8118-b05e59ee4bef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:19.729583 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6c40be-0dc6-4610-8118-b05e59ee4bef None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:19.785451 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:19.854739 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 125/244] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:05:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:05:19.858039 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b6cf69-e1b9-413e-a51f-4610e1efcd4e None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:19.899440 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:19.993156 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b6cf69-e1b9-413e-a51f-4610e1efcd4e None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:20.047179 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1062fa7f-17c3-477c-8a40-c8fd7049362e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:20.048031 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1062fa7f-17c3-477c-8a40-c8fd7049362e None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:20.048324 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1062fa7f-17c3-477c-8a40-c8fd7049362e None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:20.048632 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1062fa7f-17c3-477c-8a40-c8fd7049362e None admin] PATH_INFO: `/v3/projects/53e988a4b11648fcb59e0748d5dbf37b` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:20.053162 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1062fa7f-17c3-477c-8a40-c8fd7049362e None admin] RBAC: Authorizing `identity:get_project(project_id=53e988a4b11648fcb59e0748d5dbf37b)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:20.054648 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1062fa7f-17c3-477c-8a40-c8fd7049362e None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:20.056614 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 120/245] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jun 21 18:05:19 2022] GET /identity/v3/projects/53e988a4b11648fcb59e0748d5dbf37b => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:20.063183 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6c40be-0dc6-4610-8118-b05e59ee4bef None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:20.163809 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fceb6d-ae54-4aef-b397-6f19e7a6ff13 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:20.164209 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fceb6d-ae54-4aef-b397-6f19e7a6ff13 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:20.164527 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fceb6d-ae54-4aef-b397-6f19e7a6ff13 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:20.165923 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 121/246] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:05:20.182782 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0a3531-496f-416e-a5df-bd4f97272b07 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:20.183077 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0a3531-496f-416e-a5df-bd4f97272b07 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:20.183342 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0a3531-496f-416e-a5df-bd4f97272b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:20.205913 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6c40be-0dc6-4610-8118-b05e59ee4bef None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:20.210613 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-ef0a3531-496f-416e-a5df-bd4f97272b07 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:20.212032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-ef0a3531-496f-416e-a5df-bd4f97272b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:20.270322 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9704b9e3-158f-49c9-93ee-67e65346406e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:20.271351 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9704b9e3-158f-49c9-93ee-67e65346406e None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:20.271828 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9704b9e3-158f-49c9-93ee-67e65346406e None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:20.272322 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9704b9e3-158f-49c9-93ee-67e65346406e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:20.273406 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9704b9e3-158f-49c9-93ee-67e65346406e None admin] RBAC: Adding query filter params (role.id=55af05e4b36e44d282933b6a18958580, scope.project.id=53e988a4b11648fcb59e0748d5dbf37b, user.id=33b903b692624b40923cc6fc17ce4c17) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:20.273932 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9704b9e3-158f-49c9-93ee-67e65346406e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:20.276104 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9704b9e3-158f-49c9-93ee-67e65346406e None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:20.288813 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 126/247] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jun 21 18:05:20 2022] GET /identity/v3/role_assignments?user.id=33b903b692624b40923cc6fc17ce4c17&scope.project.id=53e988a4b11648fcb59e0748d5dbf37b&role.id=55af05e4b36e44d282933b6a18958580 => generated 259 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:20.331905 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 122/248] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:05:20.373709 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9704b9e3-158f-49c9-93ee-67e65346406e None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:20.393353 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:20.504880 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9704b9e3-158f-49c9-93ee-67e65346406e None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:20.548027 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a523fed-9e86-4135-bd86-06fe49f956f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:20.548872 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a523fed-9e86-4135-bd86-06fe49f956f3 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:20.549155 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a523fed-9e86-4135-bd86-06fe49f956f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:20.549430 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a523fed-9e86-4135-bd86-06fe49f956f3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:20.565996 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a523fed-9e86-4135-bd86-06fe49f956f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:05:20.574089 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-5a523fed-9e86-4135-bd86-06fe49f956f3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:05:20.574089 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:20.574089 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:20.574089 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:20.574089 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:20.574089 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:20.574089 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:20.574089 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:20.574089 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:20.574089 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:20.574089 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:20.574089 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:20.574089 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:05:20.574089 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:05:20.574089 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:05:20.574089 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:05:20.575117 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:05:20.575117 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:05:20.575117 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:05:20.575117 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:05:20.575117 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:20.575117 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:20.575117 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:20.575117 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:20.575117 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:20.575117 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:20.575117 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:20.575117 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:20.575117 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:20.575117 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:20.575117 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:20.575117 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:20.575932 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:20.575932 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:20.575932 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:20.575932 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:20.575932 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:05:20.575932 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:05:20.575932 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:05:20.575932 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:05:20.575932 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:05:20.575932 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:05:20.575932 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:05:20.575932 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:05:20.575932 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 127/249] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jun 21 18:05:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:20.583354 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0a3531-496f-416e-a5df-bd4f97272b07 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:20.659444 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:20.749584 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0a3531-496f-416e-a5df-bd4f97272b07 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:20.786409 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dff7c2-5636-432c-adaa-712481d06039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:20.787104 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dff7c2-5636-432c-adaa-712481d06039 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:20.787387 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dff7c2-5636-432c-adaa-712481d06039 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:20.787667 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dff7c2-5636-432c-adaa-712481d06039 None admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:20.788445 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dff7c2-5636-432c-adaa-712481d06039 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:20.788761 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dff7c2-5636-432c-adaa-712481d06039 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:20.790147 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dff7c2-5636-432c-adaa-712481d06039 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:20.800249 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 123/250] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:05:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:20.805857 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a523fed-9e86-4135-bd86-06fe49f956f3 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:20.905886 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:20.910288 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f549ce4-de07-4eea-b03e-27091954476c None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:20.911304 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f549ce4-de07-4eea-b03e-27091954476c None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:20.911787 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f549ce4-de07-4eea-b03e-27091954476c None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:20.913654 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 124/251] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:05:20.919860 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a564cc7e-0a2f-4394-ad68-231917f76166 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:20.921930 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a564cc7e-0a2f-4394-ad68-231917f76166 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:20.921930 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a564cc7e-0a2f-4394-ad68-231917f76166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:20.940954 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-a564cc7e-0a2f-4394-ad68-231917f76166 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:20.942814 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-a564cc7e-0a2f-4394-ad68-231917f76166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:20.963706 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a523fed-9e86-4135-bd86-06fe49f956f3 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:21.024329 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 125/252] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:05:21.046360 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f4288d-7107-4991-b7bd-bc301028ec57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:21.047277 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4288d-7107-4991-b7bd-bc301028ec57 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:21.047566 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4288d-7107-4991-b7bd-bc301028ec57 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:21.047844 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4288d-7107-4991-b7bd-bc301028ec57 None admin] PATH_INFO: `/v3/users/376aeb7be69e4a7bb27f6fbb7772ba41` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:21.049077 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f4288d-7107-4991-b7bd-bc301028ec57 None admin] RBAC: Authorizing `identity:get_user(user_id=376aeb7be69e4a7bb27f6fbb7772ba41)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:21.050566 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f4288d-7107-4991-b7bd-bc301028ec57 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:21.052537 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 128/253] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jun 21 18:05:20 2022] GET /identity/v3/users/376aeb7be69e4a7bb27f6fbb7772ba41 => generated 283 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:21.058183 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a564cc7e-0a2f-4394-ad68-231917f76166 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:21.071467 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f4288d-7107-4991-b7bd-bc301028ec57 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:21.100887 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:21.230971 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a564cc7e-0a2f-4394-ad68-231917f76166 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:21.289550 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9056d1-4b05-4d93-9d97-181361a5abee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:21.290217 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9056d1-4b05-4d93-9d97-181361a5abee None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:21.290503 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9056d1-4b05-4d93-9d97-181361a5abee None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:21.290784 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9056d1-4b05-4d93-9d97-181361a5abee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:21.341415 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f4288d-7107-4991-b7bd-bc301028ec57 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:21.436090 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9056d1-4b05-4d93-9d97-181361a5abee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:21.437149 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9578ec-abe9-4daf-b973-ebd605e2efc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:21.437687 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9056d1-4b05-4d93-9d97-181361a5abee None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:21.438498 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9578ec-abe9-4daf-b973-ebd605e2efc0 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:21.439007 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9578ec-abe9-4daf-b973-ebd605e2efc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:21.439531 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9578ec-abe9-4daf-b973-ebd605e2efc0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:21.471202 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:21.486155 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9578ec-abe9-4daf-b973-ebd605e2efc0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:05:21.499208 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-4a9578ec-abe9-4daf-b973-ebd605e2efc0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:05:21.499208 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:21.499208 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:21.499208 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:21.499208 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:21.499208 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:21.499208 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:21.499208 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:21.499208 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:21.499208 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:21.499208 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:21.499208 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:21.499208 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:05:21.499208 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:05:21.499208 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:05:21.499208 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:05:21.500197 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:05:21.500197 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:05:21.500197 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:05:21.500197 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:05:21.500197 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:21.500197 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:21.500197 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:21.500197 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:21.500197 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:21.500197 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:21.500197 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:21.500197 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:21.500197 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:21.500197 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:21.500197 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:21.500197 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:21.501070 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:21.501070 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:21.501070 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:21.501070 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:21.501070 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:05:21.501070 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:05:21.501070 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:05:21.501070 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:05:21.501070 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:05:21.501070 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:05:21.501070 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:05:21.501070 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:05:21.501070 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 129/254] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:05:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 430 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:21.501753 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:21.512947 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9578ec-abe9-4daf-b973-ebd605e2efc0 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:21.669614 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9578ec-abe9-4daf-b973-ebd605e2efc0 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:21.691791 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 126/255] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:05:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 634 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:05:21.701457 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9056d1-4b05-4d93-9d97-181361a5abee None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:21.738661 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f93464-ca5d-4d76-ad11-ce101fb2589d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:21.740435 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f93464-ca5d-4d76-ad11-ce101fb2589d None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:21.740435 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f93464-ca5d-4d76-ad11-ce101fb2589d None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:21.740435 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f93464-ca5d-4d76-ad11-ce101fb2589d None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:21.740682 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f93464-ca5d-4d76-ad11-ce101fb2589d None admin] RBAC: Adding query filter params (name=member) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:21.740881 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f93464-ca5d-4d76-ad11-ce101fb2589d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:21.742270 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f93464-ca5d-4d76-ad11-ce101fb2589d None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:21.761612 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 130/256] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jun 21 18:05:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:05:21.767038 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f93464-ca5d-4d76-ad11-ce101fb2589d None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:21.893326 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:21.912748 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f93464-ca5d-4d76-ad11-ce101fb2589d None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:21.915950 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9056d1-4b05-4d93-9d97-181361a5abee None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:21.948661 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fd3e84-74ef-401b-b62e-8f55f379aa6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:21.949609 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fd3e84-74ef-401b-b62e-8f55f379aa6c None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:21.949953 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fd3e84-74ef-401b-b62e-8f55f379aa6c None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:21.950259 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fd3e84-74ef-401b-b62e-8f55f379aa6c None admin] PATH_INFO: `/v3/users/3d12af51cf5b4a3eb07c7ef257ff00f6` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:21.951520 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fd3e84-74ef-401b-b62e-8f55f379aa6c None admin] RBAC: Authorizing `identity:get_user(user_id=3d12af51cf5b4a3eb07c7ef257ff00f6)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:21.953434 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fd3e84-74ef-401b-b62e-8f55f379aa6c None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:21.955131 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 131/257] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jun 21 18:05:21 2022] GET /identity/v3/users/3d12af51cf5b4a3eb07c7ef257ff00f6 => generated 297 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:05:21.956367 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93de612-9f5a-4794-8cf9-209a4ab04007 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:21.956684 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93de612-9f5a-4794-8cf9-209a4ab04007 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:21.956970 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93de612-9f5a-4794-8cf9-209a4ab04007 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:21.958215 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 132/258] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:05:21.969368 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93de612-9f5a-4794-8cf9-209a4ab04007 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:21.976948 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:21.997950 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40644437-0aa3-4cbf-aee4-268cb4cabd30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:22.000050 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40644437-0aa3-4cbf-aee4-268cb4cabd30 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:22.001219 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40644437-0aa3-4cbf-aee4-268cb4cabd30 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:22.001699 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40644437-0aa3-4cbf-aee4-268cb4cabd30 None admin] PATH_INFO: `/v3/projects/53e988a4b11648fcb59e0748d5dbf37b` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:22.004495 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40644437-0aa3-4cbf-aee4-268cb4cabd30 None admin] RBAC: Authorizing `identity:get_project(project_id=53e988a4b11648fcb59e0748d5dbf37b)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:22.006691 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40644437-0aa3-4cbf-aee4-268cb4cabd30 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:22.012599 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 127/259] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jun 21 18:05:21 2022] GET /identity/v3/projects/53e988a4b11648fcb59e0748d5dbf37b => generated 302 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:22.018737 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94eb8ed-af5f-4b92-94c9-2543012c1afe None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:22.019314 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94eb8ed-af5f-4b92-94c9-2543012c1afe None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:22.019866 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94eb8ed-af5f-4b92-94c9-2543012c1afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:22.053444 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-b94eb8ed-af5f-4b92-94c9-2543012c1afe None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:22.059047 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-b94eb8ed-af5f-4b92-94c9-2543012c1afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:22.155697 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93de612-9f5a-4794-8cf9-209a4ab04007 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:22.204778 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:22.208658 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 128/260] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:05:22.214648 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94eb8ed-af5f-4b92-94c9-2543012c1afe None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:22.215417 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3671601-45bc-4abb-a8da-09a079296794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:22.217010 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3671601-45bc-4abb-a8da-09a079296794 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:22.218337 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3671601-45bc-4abb-a8da-09a079296794 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:22.219864 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3671601-45bc-4abb-a8da-09a079296794 None admin] PATH_INFO: `/v3/projects/53e988a4b11648fcb59e0748d5dbf37b` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:22.224014 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3671601-45bc-4abb-a8da-09a079296794 None admin] RBAC: Authorizing `identity:get_project(project_id=53e988a4b11648fcb59e0748d5dbf37b)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:22.229667 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3671601-45bc-4abb-a8da-09a079296794 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:22.232100 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 133/261] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jun 21 18:05:21 2022] GET /identity/v3/projects/53e988a4b11648fcb59e0748d5dbf37b => generated 302 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:22.237438 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3671601-45bc-4abb-a8da-09a079296794 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:22.345273 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3671601-45bc-4abb-a8da-09a079296794 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:22.366578 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94eb8ed-af5f-4b92-94c9-2543012c1afe None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:22.381166 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42627742-7975-40c5-9cd8-df0f42311ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:22.382402 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42627742-7975-40c5-9cd8-df0f42311ad5 None admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:22.382886 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42627742-7975-40c5-9cd8-df0f42311ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:22.383506 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42627742-7975-40c5-9cd8-df0f42311ad5 None admin] PATH_INFO: `/v3/projects/53e988a4b11648fcb59e0748d5dbf37b/users/3d12af51cf5b4a3eb07c7ef257ff00f6/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:22.386606 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42627742-7975-40c5-9cd8-df0f42311ad5 None admin] RBAC: Authorizing `identity:create_grant(project_id=53e988a4b11648fcb59e0748d5dbf37b, user_id=3d12af51cf5b4a3eb07c7ef257ff00f6, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:22.388582 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42627742-7975-40c5-9cd8-df0f42311ad5 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:22.403673 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:22.417630 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6f9f22-5540-4df1-8bb8-4eda1da1ae1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:22.418299 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f9f22-5540-4df1-8bb8-4eda1da1ae1b None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:22.418677 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f9f22-5540-4df1-8bb8-4eda1da1ae1b None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:22.418910 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f9f22-5540-4df1-8bb8-4eda1da1ae1b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:22.419677 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6f9f22-5540-4df1-8bb8-4eda1da1ae1b None admin] RBAC: Adding query filter params (role.id=745ed9065a8143e384ebbd694b29635c, scope.project.id=53e988a4b11648fcb59e0748d5dbf37b, user.id=376aeb7be69e4a7bb27f6fbb7772ba41) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:22.419982 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6f9f22-5540-4df1-8bb8-4eda1da1ae1b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:22.421437 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6f9f22-5540-4df1-8bb8-4eda1da1ae1b None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:22.427584 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 129/262] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jun 21 18:05:22 2022] GET /identity/v3/role_assignments?user.id=376aeb7be69e4a7bb27f6fbb7772ba41&scope.project.id=53e988a4b11648fcb59e0748d5dbf37b&role.id=745ed9065a8143e384ebbd694b29635c => generated 608 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:22.428205 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6f9f22-5540-4df1-8bb8-4eda1da1ae1b None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:22.499591 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 134/263] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jun 21 18:05:22 2022] PUT /identity/v3/projects/53e988a4b11648fcb59e0748d5dbf37b/users/3d12af51cf5b4a3eb07c7ef257ff00f6/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:05:22.501134 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577d3693-3ab6-4b0a-8c51-3b966807f7f0 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:22.501428 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577d3693-3ab6-4b0a-8c51-3b966807f7f0 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:22.501705 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577d3693-3ab6-4b0a-8c51-3b966807f7f0 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:22.504109 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 135/264] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:05:22.511489 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b2cba-63b5-4b74-9fee-0d134df465a7 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:22.512038 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b2cba-63b5-4b74-9fee-0d134df465a7 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:22.512370 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b2cba-63b5-4b74-9fee-0d134df465a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:22.535138 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-0c3b2cba-63b5-4b74-9fee-0d134df465a7 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:22.536549 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-0c3b2cba-63b5-4b74-9fee-0d134df465a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:22.684819 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:22.687059 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6f9f22-5540-4df1-8bb8-4eda1da1ae1b None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:22.714051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 136/265] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:05:22.733484 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f865caf-595c-44f5-a1a8-b24ca899af0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:22.739062 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f865caf-595c-44f5-a1a8-b24ca899af0a None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:22.739062 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f865caf-595c-44f5-a1a8-b24ca899af0a None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:22.739062 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f865caf-595c-44f5-a1a8-b24ca899af0a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:22.748325 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f865caf-595c-44f5-a1a8-b24ca899af0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:05:22.757006 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-0f865caf-595c-44f5-a1a8-b24ca899af0a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:05:22.757006 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:22.757006 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:22.757006 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:22.757006 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:22.757006 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:22.757006 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:22.757006 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:22.757006 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:22.757006 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:22.757006 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:22.757006 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:22.757006 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:05:22.757006 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:05:22.757006 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:05:22.757006 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:05:22.757979 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:05:22.757979 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:05:22.757979 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:05:22.757979 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:05:22.757979 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:22.757979 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:22.757979 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:22.757979 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:22.757979 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:22.757979 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:22.757979 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:22.757979 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:22.757979 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:22.757979 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:22.757979 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:22.757979 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:22.758811 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:22.758811 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:22.758811 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:22.758811 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:22.758811 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:05:22.758811 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:05:22.758811 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:05:22.758811 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:05:22.758811 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:05:22.758811 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:05:22.758811 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:05:22.758811 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:05:22.759422 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3b2cba-63b5-4b74-9fee-0d134df465a7 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:22.759494 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 130/266] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jun 21 18:05:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:22.767166 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f865caf-595c-44f5-a1a8-b24ca899af0a None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:22.830996 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:22.908303 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f865caf-595c-44f5-a1a8-b24ca899af0a None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:22.915818 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3b2cba-63b5-4b74-9fee-0d134df465a7 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:22.958139 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f333041d-87bc-42ac-b6f2-c9851849886c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:22.959146 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f333041d-87bc-42ac-b6f2-c9851849886c None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:22.959442 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f333041d-87bc-42ac-b6f2-c9851849886c None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:22.959719 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f333041d-87bc-42ac-b6f2-c9851849886c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:22.963446 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f2301e-8254-4278-9901-e06a2a78603d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:22.964618 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2301e-8254-4278-9901-e06a2a78603d None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:22.964928 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2301e-8254-4278-9901-e06a2a78603d None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:22.965207 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2301e-8254-4278-9901-e06a2a78603d None admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:22.965986 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f2301e-8254-4278-9901-e06a2a78603d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:22.966295 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f2301e-8254-4278-9901-e06a2a78603d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:22.967711 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f2301e-8254-4278-9901-e06a2a78603d None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:22.977480 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 131/267] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jun 21 18:05:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:05:22.980205 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f333041d-87bc-42ac-b6f2-c9851849886c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:05:22.980338 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beda487-b76b-4798-89c7-4f3c4c870e62 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:22.980649 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beda487-b76b-4798-89c7-4f3c4c870e62 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:22.980929 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beda487-b76b-4798-89c7-4f3c4c870e62 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:22.982338 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 132/268] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:05:22.987091 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beda487-b76b-4798-89c7-4f3c4c870e62 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:22.988820 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-f333041d-87bc-42ac-b6f2-c9851849886c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:05:22.988820 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:22.988820 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:22.988820 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:22.988820 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:22.988820 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:22.988820 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:22.988820 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:22.988820 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:22.988820 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:22.988820 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:22.988820 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:22.988820 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:05:22.988820 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:05:22.988820 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:05:22.988820 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:05:22.989721 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:05:22.989721 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:05:22.989721 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:05:22.989721 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:05:22.989721 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:22.989721 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:22.989721 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:22.989721 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:22.989721 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:22.989721 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:22.989721 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:22.989721 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:22.989721 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:22.989721 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:22.989721 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:22.989721 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:22.990564 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:22.990564 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:22.990564 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:22.990564 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:22.990564 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:05:22.990564 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:05:22.990564 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:05:22.990564 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:05:22.990564 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:05:22.990564 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:05:22.990564 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:05:22.990564 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:05:22.991750 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 137/269] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:05:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:22.997313 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ab59d-391f-424a-9118-d82787878473 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:22.997610 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ab59d-391f-424a-9118-d82787878473 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:22.997897 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ab59d-391f-424a-9118-d82787878473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:23.021386 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-507ab59d-391f-424a-9118-d82787878473 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:23.022882 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-507ab59d-391f-424a-9118-d82787878473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:23.081037 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:23.114737 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 138/270] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:05:23.117323 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507ab59d-391f-424a-9118-d82787878473 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:23.125331 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beda487-b76b-4798-89c7-4f3c4c870e62 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:23.139094 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:23.175227 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d781772-48dd-4e9a-a69e-d510982c43b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:23.176118 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d781772-48dd-4e9a-a69e-d510982c43b0 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:23.176414 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d781772-48dd-4e9a-a69e-d510982c43b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:23.176690 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d781772-48dd-4e9a-a69e-d510982c43b0 None admin] PATH_INFO: `/v3/users/376aeb7be69e4a7bb27f6fbb7772ba41` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:23.177910 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d781772-48dd-4e9a-a69e-d510982c43b0 None admin] RBAC: Authorizing `identity:get_user(user_id=376aeb7be69e4a7bb27f6fbb7772ba41)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:23.179675 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d781772-48dd-4e9a-a69e-d510982c43b0 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:23.182588 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 133/271] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jun 21 18:05:22 2022] GET /identity/v3/users/376aeb7be69e4a7bb27f6fbb7772ba41 => generated 283 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:23.183496 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d781772-48dd-4e9a-a69e-d510982c43b0 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:23.248173 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507ab59d-391f-424a-9118-d82787878473 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:23.284258 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfacb8e-6391-4002-bb35-beb736138b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:23.284967 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfacb8e-6391-4002-bb35-beb736138b7c None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:23.285254 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfacb8e-6391-4002-bb35-beb736138b7c None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:23.285529 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfacb8e-6391-4002-bb35-beb736138b7c None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:23.286285 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfacb8e-6391-4002-bb35-beb736138b7c None admin] RBAC: Adding query filter params (name=member) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:23.286593 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfacb8e-6391-4002-bb35-beb736138b7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:23.287999 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfacb8e-6391-4002-bb35-beb736138b7c None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:23.297085 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 139/272] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jun 21 18:05:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:23.301110 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d781772-48dd-4e9a-a69e-d510982c43b0 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:23.302223 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfacb8e-6391-4002-bb35-beb736138b7c None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:23.341504 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4591504a-0edc-4ad6-9acd-5328e4c6ccb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:23.342369 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591504a-0edc-4ad6-9acd-5328e4c6ccb2 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:23.342650 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591504a-0edc-4ad6-9acd-5328e4c6ccb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:23.342947 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591504a-0edc-4ad6-9acd-5328e4c6ccb2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:23.344350 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4591504a-0edc-4ad6-9acd-5328e4c6ccb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:23.345427 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:23.346832 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4591504a-0edc-4ad6-9acd-5328e4c6ccb2 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:23.349082 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 134/273] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:05:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:23.350177 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b011f-246f-4333-855c-7d91fbf26a24 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:23.350469 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b011f-246f-4333-855c-7d91fbf26a24 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:23.350742 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b011f-246f-4333-855c-7d91fbf26a24 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:23.356626 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 135/274] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:05:23.356626 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880b011f-246f-4333-855c-7d91fbf26a24 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:23.411824 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:23.419396 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfacb8e-6391-4002-bb35-beb736138b7c None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:23.462006 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6defddbe-96b1-4d11-96f2-70db27b02d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:23.462803 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6defddbe-96b1-4d11-96f2-70db27b02d49 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:23.463090 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6defddbe-96b1-4d11-96f2-70db27b02d49 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:23.464334 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6defddbe-96b1-4d11-96f2-70db27b02d49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:23.473552 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880b011f-246f-4333-855c-7d91fbf26a24 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:23.513507 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6eb58d-4314-42c2-8e89-8b77ddd5fcc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:23.514733 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6eb58d-4314-42c2-8e89-8b77ddd5fcc1 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:23.515201 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6eb58d-4314-42c2-8e89-8b77ddd5fcc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:23.515690 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6eb58d-4314-42c2-8e89-8b77ddd5fcc1 None admin] PATH_INFO: `/v3/users/c8da20ff0a47444e96b40c86f1e2ffa0` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:23.530396 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6eb58d-4314-42c2-8e89-8b77ddd5fcc1 None admin] RBAC: Authorizing `identity:get_user(user_id=c8da20ff0a47444e96b40c86f1e2ffa0)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:23.531935 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6eb58d-4314-42c2-8e89-8b77ddd5fcc1 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:23.534033 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 136/275] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jun 21 18:05:23 2022] GET /identity/v3/users/c8da20ff0a47444e96b40c86f1e2ffa0 => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:23.534947 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6eb58d-4314-42c2-8e89-8b77ddd5fcc1 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:23.549430 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6defddbe-96b1-4d11-96f2-70db27b02d49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:23.550891 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6defddbe-96b1-4d11-96f2-70db27b02d49 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:23.597746 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:23.660474 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:23.663475 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6eb58d-4314-42c2-8e89-8b77ddd5fcc1 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:23.667706 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 140/276] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:05:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:05:23.668850 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbabb5f-1787-459e-85d7-2a8e8315d16f None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:23.669133 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbabb5f-1787-459e-85d7-2a8e8315d16f None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:23.669405 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbabb5f-1787-459e-85d7-2a8e8315d16f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:23.687093 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-ebbabb5f-1787-459e-85d7-2a8e8315d16f None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:23.688292 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-ebbabb5f-1787-459e-85d7-2a8e8315d16f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:23.707031 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397c67f9-9f0b-42e7-bc64-5b37ff8909df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:23.708315 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c67f9-9f0b-42e7-bc64-5b37ff8909df None admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:23.708789 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c67f9-9f0b-42e7-bc64-5b37ff8909df None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:23.709217 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c67f9-9f0b-42e7-bc64-5b37ff8909df None admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:23.710078 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397c67f9-9f0b-42e7-bc64-5b37ff8909df None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:23.712238 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397c67f9-9f0b-42e7-bc64-5b37ff8909df None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:23.758894 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 141/277] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:05:23.759551 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbabb5f-1787-459e-85d7-2a8e8315d16f None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:23.821640 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 137/278] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Jun 21 18:05:23 2022] POST /identity/v3/users => generated 293 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:05:23.822991 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397c67f9-9f0b-42e7-bc64-5b37ff8909df None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:23.842604 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:23.890120 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbabb5f-1787-459e-85d7-2a8e8315d16f None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:23.943921 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4322e73a-cc6d-494f-b6a6-48bfba107e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:23.944636 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322e73a-cc6d-494f-b6a6-48bfba107e10 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:23.946109 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322e73a-cc6d-494f-b6a6-48bfba107e10 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:23.948111 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322e73a-cc6d-494f-b6a6-48bfba107e10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:23.949051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4322e73a-cc6d-494f-b6a6-48bfba107e10 None admin] RBAC: Adding query filter params (role.id=c467f10d3dba4f2dbfd42068f42f74b6, user.id=c8da20ff0a47444e96b40c86f1e2ffa0, scope.system=all) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:23.949373 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4322e73a-cc6d-494f-b6a6-48bfba107e10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:23.950883 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4322e73a-cc6d-494f-b6a6-48bfba107e10 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:23.960695 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 142/279] 10.222.0.23 () {56 vars in 1308 bytes} [Tue Jun 21 18:05:23 2022] GET /identity/v3/role_assignments?user.id=c8da20ff0a47444e96b40c86f1e2ffa0&scope.system=all&role.id=c467f10d3dba4f2dbfd42068f42f74b6 => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:23.962083 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4322e73a-cc6d-494f-b6a6-48bfba107e10 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:23.968816 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397c67f9-9f0b-42e7-bc64-5b37ff8909df None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:24.015199 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130c28ca-826d-4b9d-8537-1b002793505b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:24.016422 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c28ca-826d-4b9d-8537-1b002793505b None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:24.016852 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c28ca-826d-4b9d-8537-1b002793505b None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:24.017385 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c28ca-826d-4b9d-8537-1b002793505b None admin] PATH_INFO: `/v3/projects/53e988a4b11648fcb59e0748d5dbf37b` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:24.018929 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130c28ca-826d-4b9d-8537-1b002793505b None admin] RBAC: Authorizing `identity:get_project(project_id=53e988a4b11648fcb59e0748d5dbf37b)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:24.020632 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130c28ca-826d-4b9d-8537-1b002793505b None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:24.022650 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 138/280] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jun 21 18:05:23 2022] GET /identity/v3/projects/53e988a4b11648fcb59e0748d5dbf37b => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:24.033463 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130c28ca-826d-4b9d-8537-1b002793505b None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:24.078299 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:24.134723 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4322e73a-cc6d-494f-b6a6-48bfba107e10 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:24.175789 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130c28ca-826d-4b9d-8537-1b002793505b None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:24.182516 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631f875a-1ae4-4916-a371-f8a6f7406c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:24.183647 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f875a-1ae4-4916-a371-f8a6f7406c34 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:24.184141 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f875a-1ae4-4916-a371-f8a6f7406c34 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:24.184697 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f875a-1ae4-4916-a371-f8a6f7406c34 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:24.187746 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:24.203484 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631f875a-1ae4-4916-a371-f8a6f7406c34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:05:24.211642 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-631f875a-1ae4-4916-a371-f8a6f7406c34 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 18:05:24.211642 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:24.211642 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:24.211642 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:24.211642 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:24.211642 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:24.211642 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:24.211642 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:24.211642 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:24.211642 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:24.211642 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:24.211642 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:24.211642 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:05:24.211642 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:05:24.211642 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:05:24.211642 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:05:24.212640 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:05:24.212640 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:05:24.212640 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:05:24.212640 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:05:24.212640 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:24.212640 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:24.212640 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:24.212640 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:24.212640 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:24.212640 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:24.212640 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:24.212640 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:24.212640 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:24.212640 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:24.212640 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:24.212640 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:24.213595 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:24.213595 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:24.213595 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:24.213595 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:24.213595 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:05:24.213595 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:05:24.213595 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:05:24.213595 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:05:24.213595 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:05:24.213595 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:05:24.213595 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 18:05:24.213595 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:05:24.213595 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 143/281] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:05:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:24.220834 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631f875a-1ae4-4916-a371-f8a6f7406c34 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:24.226228 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf7b449-aad5-4253-8321-b3807ca70ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:24.227208 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7b449-aad5-4253-8321-b3807ca70ae0 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:24.227700 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7b449-aad5-4253-8321-b3807ca70ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:24.228189 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7b449-aad5-4253-8321-b3807ca70ae0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:24.229897 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf7b449-aad5-4253-8321-b3807ca70ae0 None admin] RBAC: Adding query filter params (role.id=482dcc9ba0fd400cad2d193e3e895d4c, scope.project.id=53e988a4b11648fcb59e0748d5dbf37b, user.id=376aeb7be69e4a7bb27f6fbb7772ba41) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:24.230414 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf7b449-aad5-4253-8321-b3807ca70ae0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:24.232251 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf7b449-aad5-4253-8321-b3807ca70ae0 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:24.239878 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 139/282] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jun 21 18:05:24 2022] GET /identity/v3/role_assignments?user.id=376aeb7be69e4a7bb27f6fbb7772ba41&scope.project.id=53e988a4b11648fcb59e0748d5dbf37b&role.id=482dcc9ba0fd400cad2d193e3e895d4c => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:24.348619 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631f875a-1ae4-4916-a371-f8a6f7406c34 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:24.393993 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece15d06-d5ec-4965-9b81-e8bde86fc5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:24.394695 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece15d06-d5ec-4965-9b81-e8bde86fc5de None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:24.394982 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece15d06-d5ec-4965-9b81-e8bde86fc5de None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:24.395260 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece15d06-d5ec-4965-9b81-e8bde86fc5de None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:24.396046 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece15d06-d5ec-4965-9b81-e8bde86fc5de None admin] RBAC: Adding query filter params (name=reader) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:24.396354 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece15d06-d5ec-4965-9b81-e8bde86fc5de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:24.397791 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece15d06-d5ec-4965-9b81-e8bde86fc5de None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:24.409198 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 144/283] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jun 21 18:05:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:05:24.415023 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf7b449-aad5-4253-8321-b3807ca70ae0 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:24.531249 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:24.545736 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf7b449-aad5-4253-8321-b3807ca70ae0 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:24.583984 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38137633-9fc8-45fa-9626-1052182d1c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:24.584925 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38137633-9fc8-45fa-9626-1052182d1c9b None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:24.585206 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38137633-9fc8-45fa-9626-1052182d1c9b None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:24.585503 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38137633-9fc8-45fa-9626-1052182d1c9b None admin] PATH_INFO: `/v3/users/33b903b692624b40923cc6fc17ce4c17` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:24.587475 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38137633-9fc8-45fa-9626-1052182d1c9b None admin] RBAC: Authorizing `identity:get_user(user_id=33b903b692624b40923cc6fc17ce4c17)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:24.590322 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38137633-9fc8-45fa-9626-1052182d1c9b None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:24.592234 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 140/284] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jun 21 18:05:24 2022] GET /identity/v3/users/33b903b692624b40923cc6fc17ce4c17 => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:24.598364 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece15d06-d5ec-4965-9b81-e8bde86fc5de None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:24.671292 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:24.718733 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece15d06-d5ec-4965-9b81-e8bde86fc5de None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:24.754708 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5261b055-ad1f-4dcd-a7bf-342997241c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:24.755585 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5261b055-ad1f-4dcd-a7bf-342997241c42 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:24.755865 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5261b055-ad1f-4dcd-a7bf-342997241c42 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:24.756146 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5261b055-ad1f-4dcd-a7bf-342997241c42 None admin] PATH_INFO: `/v3/projects/53e988a4b11648fcb59e0748d5dbf37b` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:24.757345 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5261b055-ad1f-4dcd-a7bf-342997241c42 None admin] RBAC: Authorizing `identity:get_project(project_id=53e988a4b11648fcb59e0748d5dbf37b)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:24.758793 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5261b055-ad1f-4dcd-a7bf-342997241c42 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:24.761125 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 145/285] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jun 21 18:05:24 2022] GET /identity/v3/projects/53e988a4b11648fcb59e0748d5dbf37b => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:24.771479 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38137633-9fc8-45fa-9626-1052182d1c9b None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:24.847788 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8440898-58b3-4e66-9a40-11a0d905b9af None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:24.848203 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8440898-58b3-4e66-9a40-11a0d905b9af None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:24.848344 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8440898-58b3-4e66-9a40-11a0d905b9af None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:24.850271 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 146/286] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:05:24.859689 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b1169-5786-4474-97c6-8818f89892b7 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:24.859952 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b1169-5786-4474-97c6-8818f89892b7 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:24.860243 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b1169-5786-4474-97c6-8818f89892b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:24.894007 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-7d5b1169-5786-4474-97c6-8818f89892b7 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:24.895213 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-7d5b1169-5786-4474-97c6-8818f89892b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:24.999733 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38137633-9fc8-45fa-9626-1052182d1c9b None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:25.039503 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 147/287] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:05:25.047364 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6486f2b0-f790-4713-9869-633dc38ee68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:25.048266 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486f2b0-f790-4713-9869-633dc38ee68a None admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:25.048576 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486f2b0-f790-4713-9869-633dc38ee68a None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:25.048851 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486f2b0-f790-4713-9869-633dc38ee68a None admin] PATH_INFO: `/v3/projects/53e988a4b11648fcb59e0748d5dbf37b/users/33b903b692624b40923cc6fc17ce4c17/roles/55af05e4b36e44d282933b6a18958580` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:25.050821 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6486f2b0-f790-4713-9869-633dc38ee68a None admin] RBAC: Authorizing `identity:create_grant(project_id=53e988a4b11648fcb59e0748d5dbf37b, user_id=33b903b692624b40923cc6fc17ce4c17, role_id=55af05e4b36e44d282933b6a18958580)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:25.052331 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6486f2b0-f790-4713-9869-633dc38ee68a None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:25.068277 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5b1169-5786-4474-97c6-8818f89892b7 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:25.127858 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:25.136558 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 141/288] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jun 21 18:05:24 2022] PUT /identity/v3/projects/53e988a4b11648fcb59e0748d5dbf37b/users/33b903b692624b40923cc6fc17ce4c17/roles/55af05e4b36e44d282933b6a18958580 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:05:25.192277 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5b1169-5786-4474-97c6-8818f89892b7 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:25.253017 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909f27e3-547c-4da2-be20-22740694d1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:25.254253 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f27e3-547c-4da2-be20-22740694d1ae None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:25.254737 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f27e3-547c-4da2-be20-22740694d1ae None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:25.255290 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f27e3-547c-4da2-be20-22740694d1ae None admin] PATH_INFO: `/v3/roles/member` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:25.277180 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909f27e3-547c-4da2-be20-22740694d1ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:05:25.294703 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-909f27e3-547c-4da2-be20-22740694d1ae None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:05:25.294703 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:25.294703 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:25.294703 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:25.294703 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:25.294703 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:25.294703 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:25.294703 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:25.294703 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:25.294703 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:25.294703 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:25.294703 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:25.294703 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:05:25.294703 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:05:25.294703 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:05:25.294703 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:05:25.298018 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:05:25.298018 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:05:25.298018 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:05:25.298018 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:05:25.298018 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:25.298018 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:25.298018 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:25.298018 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:25.298018 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:25.298018 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:25.298018 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:25.298018 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:25.298018 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:25.298018 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:25.298018 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:25.298018 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:25.301315 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:25.301315 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:25.301315 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:25.301315 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:25.301315 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:05:25.301315 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:05:25.301315 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:05:25.301315 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:05:25.301315 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:05:25.301315 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:05:25.301315 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:05:25.301315 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:05:25.301315 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 148/289] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:05:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:05:25.307929 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6486f2b0-f790-4713-9869-633dc38ee68a None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:25.355746 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:25.553119 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6486f2b0-f790-4713-9869-633dc38ee68a None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:25.629423 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771e3812-a468-4ce7-be93-15a9f9e3ae11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:25.630546 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e3812-a468-4ce7-be93-15a9f9e3ae11 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:25.631019 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e3812-a468-4ce7-be93-15a9f9e3ae11 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:25.631472 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e3812-a468-4ce7-be93-15a9f9e3ae11 None admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:25.632856 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771e3812-a468-4ce7-be93-15a9f9e3ae11 None admin] RBAC: Adding query filter params (name=member) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:25.633386 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771e3812-a468-4ce7-be93-15a9f9e3ae11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:25.637314 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771e3812-a468-4ce7-be93-15a9f9e3ae11 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:25.654944 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 142/290] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jun 21 18:05:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:25.663259 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909f27e3-547c-4da2-be20-22740694d1ae None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:25.877563 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909f27e3-547c-4da2-be20-22740694d1ae None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:25.907511 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:25.950518 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a2384d-907a-43a8-b554-93f44d3c3b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:25.952123 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2384d-907a-43a8-b554-93f44d3c3b8f None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:25.952640 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2384d-907a-43a8-b554-93f44d3c3b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:25.953128 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2384d-907a-43a8-b554-93f44d3c3b8f None admin] PATH_INFO: `/v3/users/3d12af51cf5b4a3eb07c7ef257ff00f6` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:25.955545 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a2384d-907a-43a8-b554-93f44d3c3b8f None admin] RBAC: Authorizing `identity:get_user(user_id=3d12af51cf5b4a3eb07c7ef257ff00f6)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:25.958278 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a2384d-907a-43a8-b554-93f44d3c3b8f None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:25.969958 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 149/291] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jun 21 18:05:25 2022] GET /identity/v3/users/3d12af51cf5b4a3eb07c7ef257ff00f6 => generated 297 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:25.979686 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771e3812-a468-4ce7-be93-15a9f9e3ae11 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:26.148824 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:26.194796 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771e3812-a468-4ce7-be93-15a9f9e3ae11 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:26.264341 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c34a707-d9ef-43e9-98c9-7fcd4e86de07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:26.265440 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c34a707-d9ef-43e9-98c9-7fcd4e86de07 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:26.265909 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c34a707-d9ef-43e9-98c9-7fcd4e86de07 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:26.266387 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c34a707-d9ef-43e9-98c9-7fcd4e86de07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:26.387321 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c34a707-d9ef-43e9-98c9-7fcd4e86de07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:26.390065 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c34a707-d9ef-43e9-98c9-7fcd4e86de07 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:26.542429 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:26.581410 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 143/292] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:05:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 604 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:05:26.591951 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a2384d-907a-43a8-b554-93f44d3c3b8f None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:26.850127 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a2384d-907a-43a8-b554-93f44d3c3b8f None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:26.921930 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f52b16-3e9a-47d5-8b6d-cc3daf723e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:26.922878 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f52b16-3e9a-47d5-8b6d-cc3daf723e66 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:26.923194 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f52b16-3e9a-47d5-8b6d-cc3daf723e66 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:26.923516 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f52b16-3e9a-47d5-8b6d-cc3daf723e66 None admin] PATH_INFO: `/v3/projects/53e988a4b11648fcb59e0748d5dbf37b` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:26.926401 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f52b16-3e9a-47d5-8b6d-cc3daf723e66 None admin] RBAC: Authorizing `identity:get_project(project_id=53e988a4b11648fcb59e0748d5dbf37b)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:26.928903 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f52b16-3e9a-47d5-8b6d-cc3daf723e66 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:26.930732 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 150/293] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jun 21 18:05:26 2022] GET /identity/v3/projects/53e988a4b11648fcb59e0748d5dbf37b => generated 302 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:05:26.941563 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c34a707-d9ef-43e9-98c9-7fcd4e86de07 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:27.000937 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda76882-1ce6-48e0-9ab7-228fce8570ca None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:27.001265 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda76882-1ce6-48e0-9ab7-228fce8570ca None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:27.001680 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda76882-1ce6-48e0-9ab7-228fce8570ca None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:27.003373 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 151/294] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:05:27.008642 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208bfa0-890a-4892-8af2-52aecf2306fe None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:27.009020 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208bfa0-890a-4892-8af2-52aecf2306fe None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:27.009311 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208bfa0-890a-4892-8af2-52aecf2306fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:27.049463 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-7208bfa0-890a-4892-8af2-52aecf2306fe None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:27.051133 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-7208bfa0-890a-4892-8af2-52aecf2306fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:27.141419 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c34a707-d9ef-43e9-98c9-7fcd4e86de07 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:27.159099 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:27.184798 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69a23a1-03a2-4623-99a6-7ef56d3478ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:27.186149 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a23a1-03a2-4623-99a6-7ef56d3478ab None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:27.186659 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a23a1-03a2-4623-99a6-7ef56d3478ab None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:27.187131 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a23a1-03a2-4623-99a6-7ef56d3478ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:27.188170 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69a23a1-03a2-4623-99a6-7ef56d3478ab None admin] RBAC: Adding query filter params (role.id=c467f10d3dba4f2dbfd42068f42f74b6, scope.project.id=53e988a4b11648fcb59e0748d5dbf37b, user.id=3d12af51cf5b4a3eb07c7ef257ff00f6) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:27.189401 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69a23a1-03a2-4623-99a6-7ef56d3478ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:27.191040 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69a23a1-03a2-4623-99a6-7ef56d3478ab None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:27.204432 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 144/295] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jun 21 18:05:26 2022] GET /identity/v3/role_assignments?user.id=3d12af51cf5b4a3eb07c7ef257ff00f6&scope.project.id=53e988a4b11648fcb59e0748d5dbf37b&role.id=c467f10d3dba4f2dbfd42068f42f74b6 => generated 608 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:27.220301 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 152/296] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:05:27.256812 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69a23a1-03a2-4623-99a6-7ef56d3478ab None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:27.354006 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:27.382843 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3276ec6-1d73-4586-925f-f26c0d5ffe44 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:27.383137 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3276ec6-1d73-4586-925f-f26c0d5ffe44 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:27.383752 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3276ec6-1d73-4586-925f-f26c0d5ffe44 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:27.385247 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 153/297] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:05:27.387133 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff1f08-28ed-4a54-9a2f-6ba484ca2a89 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:27.387616 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff1f08-28ed-4a54-9a2f-6ba484ca2a89 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:27.388058 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff1f08-28ed-4a54-9a2f-6ba484ca2a89 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:27.389669 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 154/298] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:05:27.394926 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df49dd1-7653-4b32-a629-6a39a1103b7b None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:27.395490 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df49dd1-7653-4b32-a629-6a39a1103b7b None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:27.395965 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df49dd1-7653-4b32-a629-6a39a1103b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:27.420124 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-2df49dd1-7653-4b32-a629-6a39a1103b7b None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:27.422247 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-2df49dd1-7653-4b32-a629-6a39a1103b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:27.428759 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69a23a1-03a2-4623-99a6-7ef56d3478ab None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:27.476523 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e18191c-6735-4fdd-9c85-ca5e497cf389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:27.478206 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e18191c-6735-4fdd-9c85-ca5e497cf389 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:27.479017 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e18191c-6735-4fdd-9c85-ca5e497cf389 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:27.479415 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e18191c-6735-4fdd-9c85-ca5e497cf389 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:27.492098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e18191c-6735-4fdd-9c85-ca5e497cf389 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:05:27.500713 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-5e18191c-6735-4fdd-9c85-ca5e497cf389 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:05:27.500713 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:27.500713 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:27.500713 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:27.500713 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:27.500713 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:27.500713 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:27.500713 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:27.500713 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:27.500713 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:27.500713 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:27.500713 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:27.500713 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:05:27.500713 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:05:27.500713 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:05:27.500713 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:05:27.501816 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:05:27.501816 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:05:27.501816 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:05:27.501816 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:05:27.501816 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:27.501816 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:27.501816 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:27.501816 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:27.501816 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:27.501816 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:27.501816 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:27.501816 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:27.501816 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:27.501816 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:27.501816 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:27.501816 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:27.502850 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:27.502850 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:27.502850 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:27.502850 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:27.502850 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:05:27.502850 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:05:27.502850 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:05:27.502850 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:05:27.502850 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:05:27.502850 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:05:27.502850 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:05:27.502850 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:05:27.504942 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 145/299] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:05:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:05:27.506558 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1b711c-d652-4cd0-b5b0-5dd2e49170dd None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:27.507028 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1b711c-d652-4cd0-b5b0-5dd2e49170dd None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:27.507478 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1b711c-d652-4cd0-b5b0-5dd2e49170dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:27.508684 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 155/300] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:05:27.511828 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df49dd1-7653-4b32-a629-6a39a1103b7b None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:27.534624 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-5b1b711c-d652-4cd0-b5b0-5dd2e49170dd None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:27.536056 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-5b1b711c-d652-4cd0-b5b0-5dd2e49170dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:27.623982 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 146/301] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:05:27.625253 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1b711c-d652-4cd0-b5b0-5dd2e49170dd None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:27.637478 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df49dd1-7653-4b32-a629-6a39a1103b7b None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:27.651728 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:27.660245 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:27.677315 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090e4896-ee47-4e55-a414-7d8ba07fd223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:27.678048 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e4896-ee47-4e55-a414-7d8ba07fd223 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:27.678302 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e4896-ee47-4e55-a414-7d8ba07fd223 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:27.678553 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e4896-ee47-4e55-a414-7d8ba07fd223 None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:27.679338 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090e4896-ee47-4e55-a414-7d8ba07fd223 None admin] RBAC: Adding query filter params (name=member) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:27.679657 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090e4896-ee47-4e55-a414-7d8ba07fd223 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:27.681052 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090e4896-ee47-4e55-a414-7d8ba07fd223 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:27.694125 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 156/302] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jun 21 18:05:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:27.694125 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090e4896-ee47-4e55-a414-7d8ba07fd223 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:27.743024 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1b711c-d652-4cd0-b5b0-5dd2e49170dd None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:27.781017 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439008e4-5cc8-4555-905b-f4eaa66b4bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:27.781925 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439008e4-5cc8-4555-905b-f4eaa66b4bac None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:27.782205 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439008e4-5cc8-4555-905b-f4eaa66b4bac None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:27.782477 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439008e4-5cc8-4555-905b-f4eaa66b4bac None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:27.795505 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439008e4-5cc8-4555-905b-f4eaa66b4bac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:05:27.797223 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090e4896-ee47-4e55-a414-7d8ba07fd223 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:27.804839 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-439008e4-5cc8-4555-905b-f4eaa66b4bac None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 18:05:27.804839 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:27.804839 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:27.804839 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:27.804839 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:27.804839 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:27.804839 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:27.804839 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:27.804839 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:27.804839 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:27.804839 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:27.804839 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:27.804839 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:05:27.804839 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:05:27.804839 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:05:27.804839 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:05:27.805775 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:05:27.805775 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:05:27.805775 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:05:27.805775 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:05:27.805775 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:27.805775 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:27.805775 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:27.805775 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:27.805775 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:27.805775 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:27.805775 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:27.805775 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:27.805775 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:27.805775 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:27.805775 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:27.805775 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:27.811681 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:27.811681 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:27.811681 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:27.811681 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:27.811681 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:05:27.811681 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:05:27.811681 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:05:27.811681 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:05:27.811681 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:05:27.811681 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:05:27.811681 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 18:05:27.811681 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:05:27.811681 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 147/303] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:05:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:27.811681 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439008e4-5cc8-4555-905b-f4eaa66b4bac None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:27.831787 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae808dd-55d1-4a3b-96eb-7dd1dc05b332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:27.832588 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae808dd-55d1-4a3b-96eb-7dd1dc05b332 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:27.832910 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae808dd-55d1-4a3b-96eb-7dd1dc05b332 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:27.833155 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae808dd-55d1-4a3b-96eb-7dd1dc05b332 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:27.834544 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae808dd-55d1-4a3b-96eb-7dd1dc05b332 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:27.836051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae808dd-55d1-4a3b-96eb-7dd1dc05b332 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:27.837761 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 157/304] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:05:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:27.839204 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae808dd-55d1-4a3b-96eb-7dd1dc05b332 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:27.887624 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:27.933461 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:27.942600 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439008e4-5cc8-4555-905b-f4eaa66b4bac None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:27.950836 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae808dd-55d1-4a3b-96eb-7dd1dc05b332 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:27.980681 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c172e191-3065-4c50-868d-fdeafa3dfb62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:27.981562 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c172e191-3065-4c50-868d-fdeafa3dfb62 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:27.981977 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c172e191-3065-4c50-868d-fdeafa3dfb62 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:27.982256 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c172e191-3065-4c50-868d-fdeafa3dfb62 None admin] PATH_INFO: `/v3/users/c8da20ff0a47444e96b40c86f1e2ffa0` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:27.983449 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c172e191-3065-4c50-868d-fdeafa3dfb62 None admin] RBAC: Authorizing `identity:get_user(user_id=c8da20ff0a47444e96b40c86f1e2ffa0)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:27.984935 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c172e191-3065-4c50-868d-fdeafa3dfb62 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:27.985999 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1a29a7-2fba-4c6b-abd0-50c31779a67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:27.991371 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 148/305] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jun 21 18:05:27 2022] GET /identity/v3/users/c8da20ff0a47444e96b40c86f1e2ffa0 => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:27.991503 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1a29a7-2fba-4c6b-abd0-50c31779a67e None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:27.991503 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1a29a7-2fba-4c6b-abd0-50c31779a67e None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:27.991503 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1a29a7-2fba-4c6b-abd0-50c31779a67e None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:27.991748 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c172e191-3065-4c50-868d-fdeafa3dfb62 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:27.991822 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1a29a7-2fba-4c6b-abd0-50c31779a67e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:27.991822 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1a29a7-2fba-4c6b-abd0-50c31779a67e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:27.991822 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1a29a7-2fba-4c6b-abd0-50c31779a67e None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:28.003865 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 158/306] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jun 21 18:05:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:28.003865 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1a29a7-2fba-4c6b-abd0-50c31779a67e None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:28.109918 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c172e191-3065-4c50-868d-fdeafa3dfb62 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:28.113881 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1a29a7-2fba-4c6b-abd0-50c31779a67e None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:28.115535 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:28.146361 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157ef119-f014-42a2-b34d-35d74341fa41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:28.147386 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ef119-f014-42a2-b34d-35d74341fa41 None admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:28.147685 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ef119-f014-42a2-b34d-35d74341fa41 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:28.147966 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ef119-f014-42a2-b34d-35d74341fa41 None admin] PATH_INFO: `/v3/groups` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:28.148702 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157ef119-f014-42a2-b34d-35d74341fa41 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:28.149708 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4f02ff-95cb-4cfa-8c73-358723c969e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:28.150141 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157ef119-f014-42a2-b34d-35d74341fa41 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:28.151385 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f02ff-95cb-4cfa-8c73-358723c969e9 None admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:28.151813 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f02ff-95cb-4cfa-8c73-358723c969e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:28.152030 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f02ff-95cb-4cfa-8c73-358723c969e9 None admin] PATH_INFO: `/v3/system/users/c8da20ff0a47444e96b40c86f1e2ffa0/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:28.153607 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4f02ff-95cb-4cfa-8c73-358723c969e9 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=c8da20ff0a47444e96b40c86f1e2ffa0, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:28.154998 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4f02ff-95cb-4cfa-8c73-358723c969e9 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:28.248591 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 149/307] 10.222.0.23 () {60 vars in 1134 bytes} [Tue Jun 21 18:05:27 2022] POST /identity/v3/groups => generated 229 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:05:28.248591 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157ef119-f014-42a2-b34d-35d74341fa41 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:28.267804 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 159/308] 10.222.0.23 () {58 vars in 1334 bytes} [Tue Jun 21 18:05:27 2022] PUT /identity/v3/system/users/c8da20ff0a47444e96b40c86f1e2ffa0/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:05:28.278409 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:28.290510 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdacee80-1066-4af3-92d6-8bf517596e7e None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:28.290895 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdacee80-1066-4af3-92d6-8bf517596e7e None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:28.291132 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdacee80-1066-4af3-92d6-8bf517596e7e None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:28.292769 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 160/309] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:05:28.298880 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1945e1a0-0984-46c9-998c-acd412b1821b None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:28.299240 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1945e1a0-0984-46c9-998c-acd412b1821b None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:28.299453 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1945e1a0-0984-46c9-998c-acd412b1821b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:28.320331 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-1945e1a0-0984-46c9-998c-acd412b1821b None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:28.323586 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-1945e1a0-0984-46c9-998c-acd412b1821b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:28.380885 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157ef119-f014-42a2-b34d-35d74341fa41 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:28.418453 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 161/310] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:05:28.423725 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d720e7-a6f3-4fbb-a856-6316b1840906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:28.425042 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d720e7-a6f3-4fbb-a856-6316b1840906 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:28.425544 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d720e7-a6f3-4fbb-a856-6316b1840906 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:28.426083 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d720e7-a6f3-4fbb-a856-6316b1840906 None admin] PATH_INFO: `/v3/users/6a2dc128e93842c894d499be281ed626` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:28.443272 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d720e7-a6f3-4fbb-a856-6316b1840906 None admin] RBAC: Authorizing `identity:get_user(user_id=6a2dc128e93842c894d499be281ed626)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:28.443877 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1945e1a0-0984-46c9-998c-acd412b1821b None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:28.445478 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d720e7-a6f3-4fbb-a856-6316b1840906 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:28.449374 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 150/311] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jun 21 18:05:28 2022] GET /identity/v3/users/6a2dc128e93842c894d499be281ed626 => generated 293 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:28.455315 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d720e7-a6f3-4fbb-a856-6316b1840906 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:28.544519 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:28.565569 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:28.568870 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d720e7-a6f3-4fbb-a856-6316b1840906 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:28.573096 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1945e1a0-0984-46c9-998c-acd412b1821b None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:28.613409 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d48a366-4f86-4d35-8398-0c4585549e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:28.614082 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d48a366-4f86-4d35-8398-0c4585549e0d None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:28.614368 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d48a366-4f86-4d35-8398-0c4585549e0d None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:28.614646 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d48a366-4f86-4d35-8398-0c4585549e0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:28.615696 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d48a366-4f86-4d35-8398-0c4585549e0d None admin] RBAC: Adding query filter params (role.id=55af05e4b36e44d282933b6a18958580, user.id=6a2dc128e93842c894d499be281ed626, scope.system=all) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:28.616009 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d48a366-4f86-4d35-8398-0c4585549e0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:28.616784 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a57f4e3-ef55-42f0-866d-bbb2e14101b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:28.617480 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d48a366-4f86-4d35-8398-0c4585549e0d None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:28.617913 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57f4e3-ef55-42f0-866d-bbb2e14101b8 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:28.618341 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57f4e3-ef55-42f0-866d-bbb2e14101b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:28.618757 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57f4e3-ef55-42f0-866d-bbb2e14101b8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:28.623971 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 151/312] 10.222.0.23 () {56 vars in 1308 bytes} [Tue Jun 21 18:05:28 2022] GET /identity/v3/role_assignments?user.id=6a2dc128e93842c894d499be281ed626&scope.system=all&role.id=55af05e4b36e44d282933b6a18958580 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:05:28.637339 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a57f4e3-ef55-42f0-866d-bbb2e14101b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:05:28.646238 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-6a57f4e3-ef55-42f0-866d-bbb2e14101b8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 18:05:28.646238 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:28.646238 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:28.646238 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:28.646238 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:28.646238 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:28.646238 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:28.646238 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:28.646238 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:28.646238 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:28.646238 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:28.646238 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:28.646238 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:05:28.646238 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:05:28.646238 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:05:28.646238 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:05:28.647452 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:05:28.647452 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:05:28.647452 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:05:28.647452 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:05:28.647452 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:28.647452 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:28.647452 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:28.647452 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:28.647452 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:28.647452 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:28.647452 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:28.647452 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:28.647452 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:28.647452 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:28.647452 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:28.647452 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:28.648626 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:28.648626 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:28.648626 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:28.648626 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:28.648626 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:05:28.648626 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:05:28.648626 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:05:28.648626 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:05:28.648626 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:05:28.648626 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:05:28.648626 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 18:05:28.648626 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:05:28.651246 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 162/313] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:05:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:05:28.657341 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d48a366-4f86-4d35-8398-0c4585549e0d None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:28.772855 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d48a366-4f86-4d35-8398-0c4585549e0d None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:28.809977 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9716a8f4-1204-4ff5-aca9-970934153b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:28.810587 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9716a8f4-1204-4ff5-aca9-970934153b18 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:28.810873 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9716a8f4-1204-4ff5-aca9-970934153b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:28.811153 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9716a8f4-1204-4ff5-aca9-970934153b18 None admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:28.811913 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9716a8f4-1204-4ff5-aca9-970934153b18 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:28.812226 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9716a8f4-1204-4ff5-aca9-970934153b18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:28.813639 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9716a8f4-1204-4ff5-aca9-970934153b18 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:28.822686 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 152/314] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jun 21 18:05:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:28.828458 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a57f4e3-ef55-42f0-866d-bbb2e14101b8 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:28.940822 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a57f4e3-ef55-42f0-866d-bbb2e14101b8 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:28.975935 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07bb054-96ad-4ed6-946b-09c6736b1266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:28.976845 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07bb054-96ad-4ed6-946b-09c6736b1266 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:28.977131 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07bb054-96ad-4ed6-946b-09c6736b1266 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:28.977411 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07bb054-96ad-4ed6-946b-09c6736b1266 None admin] PATH_INFO: `/v3/users/33b903b692624b40923cc6fc17ce4c17` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:28.979182 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07bb054-96ad-4ed6-946b-09c6736b1266 None admin] RBAC: Authorizing `identity:get_user(user_id=33b903b692624b40923cc6fc17ce4c17)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:28.980653 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07bb054-96ad-4ed6-946b-09c6736b1266 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:28.982617 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 163/315] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jun 21 18:05:28 2022] GET /identity/v3/users/33b903b692624b40923cc6fc17ce4c17 => generated 297 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:28.988096 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9716a8f4-1204-4ff5-aca9-970934153b18 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:28.993194 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:29.105757 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9716a8f4-1204-4ff5-aca9-970934153b18 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:29.143163 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f3458f-d907-4a8d-b23c-62627c07b4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:29.143982 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f3458f-d907-4a8d-b23c-62627c07b4c6 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:29.144264 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f3458f-d907-4a8d-b23c-62627c07b4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:29.144570 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f3458f-d907-4a8d-b23c-62627c07b4c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:29.218164 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f3458f-d907-4a8d-b23c-62627c07b4c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:29.219659 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f3458f-d907-4a8d-b23c-62627c07b4c6 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:29.222381 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:29.327912 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 153/316] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:05:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:05:29.333568 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07bb054-96ad-4ed6-946b-09c6736b1266 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:29.360866 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:29.438676 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07bb054-96ad-4ed6-946b-09c6736b1266 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:29.479660 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fa458b-d3c6-4cc0-92bb-88d5e14bdb6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:29.481022 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa458b-d3c6-4cc0-92bb-88d5e14bdb6d None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:29.481523 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa458b-d3c6-4cc0-92bb-88d5e14bdb6d None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:29.482006 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa458b-d3c6-4cc0-92bb-88d5e14bdb6d None admin] PATH_INFO: `/v3/projects/53e988a4b11648fcb59e0748d5dbf37b` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:29.483596 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fa458b-d3c6-4cc0-92bb-88d5e14bdb6d None admin] RBAC: Authorizing `identity:get_project(project_id=53e988a4b11648fcb59e0748d5dbf37b)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:29.485373 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fa458b-d3c6-4cc0-92bb-88d5e14bdb6d None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:29.487794 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 164/317] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jun 21 18:05:29 2022] GET /identity/v3/projects/53e988a4b11648fcb59e0748d5dbf37b => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:29.496450 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f3458f-d907-4a8d-b23c-62627c07b4c6 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:29.602325 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f3458f-d907-4a8d-b23c-62627c07b4c6 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:29.605301 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:29.640178 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddbd3ca-2118-4161-a477-7ad1b6ad6906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:29.640794 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddbd3ca-2118-4161-a477-7ad1b6ad6906 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:29.641139 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddbd3ca-2118-4161-a477-7ad1b6ad6906 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:29.641420 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddbd3ca-2118-4161-a477-7ad1b6ad6906 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:29.642417 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddbd3ca-2118-4161-a477-7ad1b6ad6906 None admin] RBAC: Adding query filter params (role.id=55af05e4b36e44d282933b6a18958580, scope.project.id=53e988a4b11648fcb59e0748d5dbf37b, user.id=33b903b692624b40923cc6fc17ce4c17) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:29.642736 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddbd3ca-2118-4161-a477-7ad1b6ad6906 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:29.644357 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddbd3ca-2118-4161-a477-7ad1b6ad6906 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:29.651387 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 154/318] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jun 21 18:05:29 2022] GET /identity/v3/role_assignments?user.id=33b903b692624b40923cc6fc17ce4c17&scope.project.id=53e988a4b11648fcb59e0748d5dbf37b&role.id=55af05e4b36e44d282933b6a18958580 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:30.433388 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74079089-ca42-4066-9138-8f698d6108ee None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:30.433944 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74079089-ca42-4066-9138-8f698d6108ee None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:30.433944 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74079089-ca42-4066-9138-8f698d6108ee None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:30.435484 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 165/319] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:05:30.440705 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffff2bf-85ef-45f1-a6e0-3eb665fcf510 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:30.441112 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffff2bf-85ef-45f1-a6e0-3eb665fcf510 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:30.441309 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffff2bf-85ef-45f1-a6e0-3eb665fcf510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:30.458799 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-9ffff2bf-85ef-45f1-a6e0-3eb665fcf510 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:30.459980 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-9ffff2bf-85ef-45f1-a6e0-3eb665fcf510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:30.481075 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4691f26-b1d1-413c-a920-b3b6bf23e7d9 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:30.481369 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4691f26-b1d1-413c-a920-b3b6bf23e7d9 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:30.481647 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4691f26-b1d1-413c-a920-b3b6bf23e7d9 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:30.483732 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 166/320] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:05:30.488772 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547dbb9-6405-45e7-bbee-5919a84d0e53 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:30.489068 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547dbb9-6405-45e7-bbee-5919a84d0e53 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:30.489355 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547dbb9-6405-45e7-bbee-5919a84d0e53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:30.508845 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-6547dbb9-6405-45e7-bbee-5919a84d0e53 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:30.511632 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-6547dbb9-6405-45e7-bbee-5919a84d0e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:30.530297 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 155/321] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:05:30.554158 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffff2bf-85ef-45f1-a6e0-3eb665fcf510 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:30.589961 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 167/322] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:05:30.613412 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6547dbb9-6405-45e7-bbee-5919a84d0e53 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:30.639610 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:30.658216 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:30.662480 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffff2bf-85ef-45f1-a6e0-3eb665fcf510 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:30.700024 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eeb91b-2ae2-481b-bf5b-d48f91d933c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:30.700780 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eeb91b-2ae2-481b-bf5b-d48f91d933c6 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:30.701065 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eeb91b-2ae2-481b-bf5b-d48f91d933c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:30.701342 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eeb91b-2ae2-481b-bf5b-d48f91d933c6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:30.711566 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eeb91b-2ae2-481b-bf5b-d48f91d933c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:05:30.715051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6547dbb9-6405-45e7-bbee-5919a84d0e53 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:30.720580 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-03eeb91b-2ae2-481b-bf5b-d48f91d933c6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:05:30.720580 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:30.720580 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:30.720580 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:30.720580 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:30.720580 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:30.720580 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:30.720580 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:30.720580 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:30.720580 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:30.720580 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:30.720580 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:30.720580 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:05:30.720580 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:05:30.720580 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:05:30.720580 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:05:30.721660 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:05:30.721660 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:05:30.721660 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:05:30.721660 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:05:30.721660 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:30.721660 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:30.721660 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:30.721660 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:30.721660 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:30.721660 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:30.721660 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:30.721660 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:30.721660 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:30.721660 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:30.721660 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:30.721660 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:30.722804 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:30.722804 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:30.722804 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:30.722804 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:30.722804 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:05:30.722804 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:05:30.722804 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:05:30.722804 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:05:30.722804 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:05:30.722804 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:05:30.722804 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:05:30.722804 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:05:30.723766 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 156/323] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:05:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:05:30.724270 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf330a4-a0de-441f-8c41-1cedfd0e2703 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:30.724481 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf330a4-a0de-441f-8c41-1cedfd0e2703 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:30.724748 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf330a4-a0de-441f-8c41-1cedfd0e2703 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:30.726162 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 157/324] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:05:30.728827 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf330a4-a0de-441f-8c41-1cedfd0e2703 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:30.751673 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20854875-54ba-48fa-98c0-182036e7ec53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:30.752500 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20854875-54ba-48fa-98c0-182036e7ec53 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:30.752785 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20854875-54ba-48fa-98c0-182036e7ec53 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:30.753057 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20854875-54ba-48fa-98c0-182036e7ec53 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:30.754212 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20854875-54ba-48fa-98c0-182036e7ec53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:30.755638 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20854875-54ba-48fa-98c0-182036e7ec53 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:30.757468 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 168/325] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:05:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:05:30.758402 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50758275-98c0-449c-a958-6233e6e0d1ec None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:30.758688 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50758275-98c0-449c-a958-6233e6e0d1ec None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:30.758963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50758275-98c0-449c-a958-6233e6e0d1ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:30.774652 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-50758275-98c0-449c-a958-6233e6e0d1ec None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:30.776760 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-50758275-98c0-449c-a958-6233e6e0d1ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:30.831228 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf330a4-a0de-441f-8c41-1cedfd0e2703 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:30.842561 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 169/326] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:05:30.851750 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50758275-98c0-449c-a958-6233e6e0d1ec None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:30.867795 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3140af4-ee99-4dbf-91bb-f123f8497e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:30.868506 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3140af4-ee99-4dbf-91bb-f123f8497e4e None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:30.868795 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3140af4-ee99-4dbf-91bb-f123f8497e4e None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:30.869627 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3140af4-ee99-4dbf-91bb-f123f8497e4e None admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:30.870388 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3140af4-ee99-4dbf-91bb-f123f8497e4e None admin] RBAC: Adding query filter params (name=member) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:30.870697 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3140af4-ee99-4dbf-91bb-f123f8497e4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:30.872150 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3140af4-ee99-4dbf-91bb-f123f8497e4e None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:30.874940 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:30.885831 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 158/327] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jun 21 18:05:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:05:30.885831 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3140af4-ee99-4dbf-91bb-f123f8497e4e None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:30.903678 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:30.962328 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50758275-98c0-449c-a958-6233e6e0d1ec None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:30.981411 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3140af4-ee99-4dbf-91bb-f123f8497e4e None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:30.997424 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2646e217-3c2f-4432-ba22-127c11a797c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:30.998127 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2646e217-3c2f-4432-ba22-127c11a797c8 None admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:30.998417 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2646e217-3c2f-4432-ba22-127c11a797c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:30.998697 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2646e217-3c2f-4432-ba22-127c11a797c8 None admin] PATH_INFO: `/v3/groups` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:30.999391 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2646e217-3c2f-4432-ba22-127c11a797c8 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:31.000914 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2646e217-3c2f-4432-ba22-127c11a797c8 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:31.015507 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d292a1bd-d484-494a-a35e-5820af83e07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:31.016303 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d292a1bd-d484-494a-a35e-5820af83e07a None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:31.016613 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d292a1bd-d484-494a-a35e-5820af83e07a None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:31.016886 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d292a1bd-d484-494a-a35e-5820af83e07a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:31.027126 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d292a1bd-d484-494a-a35e-5820af83e07a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:05:31.034866 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-d292a1bd-d484-494a-a35e-5820af83e07a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 18:05:31.034866 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:31.034866 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:31.034866 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:31.034866 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:31.034866 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:31.034866 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:31.034866 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:31.034866 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:31.034866 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:31.034866 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:31.034866 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:31.034866 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:05:31.034866 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:05:31.034866 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:05:31.034866 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:05:31.036575 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:05:31.036575 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:05:31.036575 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:05:31.036575 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:05:31.036575 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:31.036575 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:31.036575 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:31.036575 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:31.036575 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:31.036575 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:31.036575 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:31.036575 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:31.036575 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:31.036575 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:31.036575 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:31.036575 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:31.038351 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:31.038351 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:31.038351 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:31.038351 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:31.038351 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:05:31.038351 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:05:31.038351 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:05:31.038351 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:05:31.038351 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:05:31.038351 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:05:31.038351 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 18:05:31.038351 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:05:31.038351 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 159/328] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:05:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:31.038351 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d292a1bd-d484-494a-a35e-5820af83e07a None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:31.095506 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 170/329] 10.222.0.23 () {60 vars in 1134 bytes} [Tue Jun 21 18:05:30 2022] POST /identity/v3/groups => generated 226 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:05:31.097338 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2646e217-3c2f-4432-ba22-127c11a797c8 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:31.126955 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:31.147635 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d292a1bd-d484-494a-a35e-5820af83e07a None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:31.191763 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90878a86-45c4-4300-9306-2d8306d1ddfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:31.193245 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90878a86-45c4-4300-9306-2d8306d1ddfd None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:31.193534 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90878a86-45c4-4300-9306-2d8306d1ddfd None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:31.193813 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90878a86-45c4-4300-9306-2d8306d1ddfd None admin] PATH_INFO: `/v3/users/c8da20ff0a47444e96b40c86f1e2ffa0` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:31.195483 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90878a86-45c4-4300-9306-2d8306d1ddfd None admin] RBAC: Authorizing `identity:get_user(user_id=c8da20ff0a47444e96b40c86f1e2ffa0)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:31.198208 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90878a86-45c4-4300-9306-2d8306d1ddfd None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:31.199538 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:31.200432 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 160/330] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jun 21 18:05:31 2022] GET /identity/v3/users/c8da20ff0a47444e96b40c86f1e2ffa0 => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:31.206089 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90878a86-45c4-4300-9306-2d8306d1ddfd None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:31.222441 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2646e217-3c2f-4432-ba22-127c11a797c8 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:31.262929 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4638f19-97a4-4b33-9f5e-6d89e05013a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:31.263615 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4638f19-97a4-4b33-9f5e-6d89e05013a0 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:31.263905 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4638f19-97a4-4b33-9f5e-6d89e05013a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:31.264234 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4638f19-97a4-4b33-9f5e-6d89e05013a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:31.265107 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4638f19-97a4-4b33-9f5e-6d89e05013a0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:31.265423 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4638f19-97a4-4b33-9f5e-6d89e05013a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:31.266813 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4638f19-97a4-4b33-9f5e-6d89e05013a0 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:31.277267 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 171/331] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jun 21 18:05:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:31.283030 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4638f19-97a4-4b33-9f5e-6d89e05013a0 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:31.331660 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90878a86-45c4-4300-9306-2d8306d1ddfd None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:31.379702 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4773cc5-e6c0-4cf6-9987-00d7a8485cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:31.381055 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4773cc5-e6c0-4cf6-9987-00d7a8485cfc None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:31.381055 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4773cc5-e6c0-4cf6-9987-00d7a8485cfc None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:31.381055 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4773cc5-e6c0-4cf6-9987-00d7a8485cfc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:31.381814 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4773cc5-e6c0-4cf6-9987-00d7a8485cfc None admin] RBAC: Adding query filter params (role.id=c467f10d3dba4f2dbfd42068f42f74b6, user.id=c8da20ff0a47444e96b40c86f1e2ffa0, scope.system=all) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:31.382130 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4773cc5-e6c0-4cf6-9987-00d7a8485cfc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:31.383589 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4773cc5-e6c0-4cf6-9987-00d7a8485cfc None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:31.397244 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 161/332] 10.222.0.23 () {56 vars in 1308 bytes} [Tue Jun 21 18:05:31 2022] GET /identity/v3/role_assignments?user.id=c8da20ff0a47444e96b40c86f1e2ffa0&scope.system=all&role.id=c467f10d3dba4f2dbfd42068f42f74b6 => generated 510 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:05:31.416927 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4638f19-97a4-4b33-9f5e-6d89e05013a0 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:31.466765 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b115d6-80f1-4118-b792-14b666984a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:31.467676 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b115d6-80f1-4118-b792-14b666984a58 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:31.467964 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b115d6-80f1-4118-b792-14b666984a58 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:31.468244 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b115d6-80f1-4118-b792-14b666984a58 None admin] PATH_INFO: `/v3/users/6a2dc128e93842c894d499be281ed626` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:31.469530 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b115d6-80f1-4118-b792-14b666984a58 None admin] RBAC: Authorizing `identity:get_user(user_id=6a2dc128e93842c894d499be281ed626)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:31.471003 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b115d6-80f1-4118-b792-14b666984a58 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:31.473693 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 172/333] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jun 21 18:05:31 2022] GET /identity/v3/users/6a2dc128e93842c894d499be281ed626 => generated 293 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:05:31.479822 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4773cc5-e6c0-4cf6-9987-00d7a8485cfc None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:31.491772 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:31.602872 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4773cc5-e6c0-4cf6-9987-00d7a8485cfc None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:31.639197 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40967b09-6a62-4912-9036-3f4f74948857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:31.640093 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40967b09-6a62-4912-9036-3f4f74948857 None admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:31.640381 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40967b09-6a62-4912-9036-3f4f74948857 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:31.640690 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40967b09-6a62-4912-9036-3f4f74948857 None admin] PATH_INFO: `/v3/system/users/6a2dc128e93842c894d499be281ed626/roles/55af05e4b36e44d282933b6a18958580` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:31.642223 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40967b09-6a62-4912-9036-3f4f74948857 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=6a2dc128e93842c894d499be281ed626, role_id=55af05e4b36e44d282933b6a18958580)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:31.643611 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40967b09-6a62-4912-9036-3f4f74948857 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:31.719816 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 162/334] 10.222.0.23 () {58 vars in 1334 bytes} [Tue Jun 21 18:05:31 2022] PUT /identity/v3/system/users/6a2dc128e93842c894d499be281ed626/roles/55af05e4b36e44d282933b6a18958580 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:05:33.638275 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642fae05-216e-4a13-979e-da981436cc85 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:33.639344 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642fae05-216e-4a13-979e-da981436cc85 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:33.639841 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642fae05-216e-4a13-979e-da981436cc85 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:33.641798 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 173/335] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:05:33.647713 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c290d5bd-32cc-4076-b089-19371c7976cb None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:33.648642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c290d5bd-32cc-4076-b089-19371c7976cb None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:33.649137 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c290d5bd-32cc-4076-b089-19371c7976cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:33.672423 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-c290d5bd-32cc-4076-b089-19371c7976cb None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:33.673986 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-c290d5bd-32cc-4076-b089-19371c7976cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:33.754391 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:33.760882 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 163/336] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:05:33.798293 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642fae05-216e-4a13-979e-da981436cc85 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:33.812210 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:33.934067 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642fae05-216e-4a13-979e-da981436cc85 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:33.984952 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5396d78d-3544-4f20-964e-189e5b59e59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:33.989290 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396d78d-3544-4f20-964e-189e5b59e59b None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:33.989582 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396d78d-3544-4f20-964e-189e5b59e59b None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:33.989863 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396d78d-3544-4f20-964e-189e5b59e59b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:34.004058 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5396d78d-3544-4f20-964e-189e5b59e59b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:05:34.012250 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-5396d78d-3544-4f20-964e-189e5b59e59b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:05:34.012250 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:34.012250 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:34.012250 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:34.012250 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:34.012250 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:34.012250 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:34.012250 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:34.012250 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:34.012250 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:34.012250 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:34.012250 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:34.012250 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:05:34.012250 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:05:34.012250 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:05:34.012250 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:05:34.013310 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:05:34.013310 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:05:34.013310 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:05:34.013310 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:05:34.013310 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:34.013310 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:34.013310 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:34.013310 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:34.013310 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:34.013310 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:34.013310 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:34.013310 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:34.013310 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:34.013310 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:34.013310 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:34.013310 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:34.014617 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:34.014617 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:34.014617 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:34.014617 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:34.014617 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:05:34.014617 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:05:34.014617 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:05:34.014617 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:05:34.014617 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:05:34.014617 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:05:34.014617 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:05:34.014617 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:05:34.014617 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 174/337] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:05:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:34.019548 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c290d5bd-32cc-4076-b089-19371c7976cb None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:34.145107 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c290d5bd-32cc-4076-b089-19371c7976cb None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:34.185006 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874f362f-8928-4dcd-9f2e-cc6d0e8ec850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:34.185608 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f362f-8928-4dcd-9f2e-cc6d0e8ec850 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:34.185895 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f362f-8928-4dcd-9f2e-cc6d0e8ec850 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:34.186176 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f362f-8928-4dcd-9f2e-cc6d0e8ec850 None admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:34.186927 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874f362f-8928-4dcd-9f2e-cc6d0e8ec850 None admin] RBAC: Adding query filter params (name=member) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:34.187233 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874f362f-8928-4dcd-9f2e-cc6d0e8ec850 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:34.188644 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874f362f-8928-4dcd-9f2e-cc6d0e8ec850 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:34.200705 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 164/338] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jun 21 18:05:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:34.206195 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5396d78d-3544-4f20-964e-189e5b59e59b None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:34.287398 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:34.325668 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5396d78d-3544-4f20-964e-189e5b59e59b None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:34.365113 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3922c2-073a-477d-9e8d-8bc08736c187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:34.365770 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3922c2-073a-477d-9e8d-8bc08736c187 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:34.366054 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3922c2-073a-477d-9e8d-8bc08736c187 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:34.366332 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3922c2-073a-477d-9e8d-8bc08736c187 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:34.451764 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3922c2-073a-477d-9e8d-8bc08736c187 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:34.453189 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3922c2-073a-477d-9e8d-8bc08736c187 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:34.536966 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:34.553181 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae5c32-b86c-4039-80f0-54282adc7297 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:34.553514 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae5c32-b86c-4039-80f0-54282adc7297 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:34.553743 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae5c32-b86c-4039-80f0-54282adc7297 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:34.555479 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 165/339] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:05:34.562549 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa11d9-48cd-4ef9-84a5-d6117ca627fc None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:34.562838 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa11d9-48cd-4ef9-84a5-d6117ca627fc None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:34.563124 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa11d9-48cd-4ef9-84a5-d6117ca627fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:34.577153 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 175/340] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:05:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:05:34.589646 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3922c2-073a-477d-9e8d-8bc08736c187 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:34.592469 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-f7fa11d9-48cd-4ef9-84a5-d6117ca627fc None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:34.593668 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-f7fa11d9-48cd-4ef9-84a5-d6117ca627fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:34.754794 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:34.767334 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3922c2-073a-477d-9e8d-8bc08736c187 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:34.810200 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 166/341] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:05:34.820647 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c6159b-fcf6-4600-bb0a-2ef0f0d7977a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:34.821528 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c6159b-fcf6-4600-bb0a-2ef0f0d7977a None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:34.821812 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c6159b-fcf6-4600-bb0a-2ef0f0d7977a None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:34.822122 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c6159b-fcf6-4600-bb0a-2ef0f0d7977a None admin] PATH_INFO: `/v3/projects/21cd4117dcd346b8bcb3e353ff8d228a` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:34.826020 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc057c3-b6b6-46ad-83ee-9be67e4b8bdd None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:34.826313 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc057c3-b6b6-46ad-83ee-9be67e4b8bdd None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:34.826593 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc057c3-b6b6-46ad-83ee-9be67e4b8bdd None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:34.828344 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 167/342] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:05:34.829240 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c6159b-fcf6-4600-bb0a-2ef0f0d7977a None admin] RBAC: Authorizing `identity:get_project(project_id=21cd4117dcd346b8bcb3e353ff8d228a)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:34.830741 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c6159b-fcf6-4600-bb0a-2ef0f0d7977a None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:34.832880 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 176/343] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jun 21 18:05:34 2022] GET /identity/v3/projects/21cd4117dcd346b8bcb3e353ff8d228a => generated 298 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:34.834145 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1120c523-69c9-42f3-988f-9658081959cb None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:34.834443 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1120c523-69c9-42f3-988f-9658081959cb None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:34.834728 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1120c523-69c9-42f3-988f-9658081959cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:34.838024 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3aa4e-e4a2-4d38-8194-2c72b05f1968 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:34.838460 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3aa4e-e4a2-4d38-8194-2c72b05f1968 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:34.838841 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3aa4e-e4a2-4d38-8194-2c72b05f1968 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:34.840194 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 177/344] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:05:34.840946 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e3aa4e-e4a2-4d38-8194-2c72b05f1968 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:34.868090 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-1120c523-69c9-42f3-988f-9658081959cb None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:34.869959 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-1120c523-69c9-42f3-988f-9658081959cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:34.903208 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:34.972084 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 168/345] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:05:34.973561 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1120c523-69c9-42f3-988f-9658081959cb None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:34.978122 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e3aa4e-e4a2-4d38-8194-2c72b05f1968 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:35.022179 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286ac0b4-28db-4600-9da9-be90043c07e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:35.022962 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ac0b4-28db-4600-9da9-be90043c07e3 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:35.023245 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ac0b4-28db-4600-9da9-be90043c07e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:35.023522 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ac0b4-28db-4600-9da9-be90043c07e3 None admin] PATH_INFO: `/v3/groups/3d53537d1bfe4cf0bc60529db50c1c6a` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:35.030098 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286ac0b4-28db-4600-9da9-be90043c07e3 None admin] RBAC: Authorizing `identity:get_group(group_id=3d53537d1bfe4cf0bc60529db50c1c6a)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:35.031548 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286ac0b4-28db-4600-9da9-be90043c07e3 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:35.033430 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 178/346] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Jun 21 18:05:34 2022] GET /identity/v3/groups/3d53537d1bfe4cf0bc60529db50c1c6a => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:35.034492 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da7c4ee-37cf-4f60-9676-024643a1ef8e None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:35.035020 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da7c4ee-37cf-4f60-9676-024643a1ef8e None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:35.035526 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da7c4ee-37cf-4f60-9676-024643a1ef8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:35.052714 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-1da7c4ee-37cf-4f60-9676-024643a1ef8e None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:35.054274 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-1da7c4ee-37cf-4f60-9676-024643a1ef8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:35.093792 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1120c523-69c9-42f3-988f-9658081959cb None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:35.121017 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:35.135564 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a378fcba-c763-4189-a9a9-2b96dac882b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:35.137476 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a378fcba-c763-4189-a9a9-2b96dac882b3 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:35.138008 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a378fcba-c763-4189-a9a9-2b96dac882b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:35.138508 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a378fcba-c763-4189-a9a9-2b96dac882b3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:35.143562 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 179/347] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:05:35.153299 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29662f17-e3c2-4db8-975b-3939b43f9838 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:35.153299 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29662f17-e3c2-4db8-975b-3939b43f9838 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:35.153299 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29662f17-e3c2-4db8-975b-3939b43f9838 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:35.153299 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 180/348] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:05:35.153299 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29662f17-e3c2-4db8-975b-3939b43f9838 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:35.159007 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a378fcba-c763-4189-a9a9-2b96dac882b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:05:35.164451 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-a378fcba-c763-4189-a9a9-2b96dac882b3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:05:35.164451 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:35.164451 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:35.164451 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:35.164451 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:35.164451 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:35.164451 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:35.164451 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:35.164451 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:35.164451 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:35.164451 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:35.164451 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:35.164451 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:05:35.164451 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:05:35.164451 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:05:35.164451 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:05:35.165413 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:05:35.165413 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:05:35.165413 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:05:35.165413 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:05:35.165413 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:35.165413 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:35.165413 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:35.165413 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:35.165413 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:35.165413 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:35.165413 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:35.165413 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:35.165413 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:35.165413 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:35.165413 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:35.165413 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:35.167116 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:35.167116 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:35.167116 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:35.167116 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:35.167116 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:05:35.167116 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:05:35.167116 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:05:35.167116 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:05:35.167116 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:05:35.167116 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:05:35.167116 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:05:35.167116 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:05:35.167116 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 169/349] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jun 21 18:05:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:35.168471 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a378fcba-c763-4189-a9a9-2b96dac882b3 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:35.194187 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:35.286253 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29662f17-e3c2-4db8-975b-3939b43f9838 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:35.298733 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a378fcba-c763-4189-a9a9-2b96dac882b3 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:35.330053 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eebb45-74ee-4336-95c8-d3c6b5bd8f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:35.331660 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eebb45-74ee-4336-95c8-d3c6b5bd8f1d None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:35.332361 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eebb45-74ee-4336-95c8-d3c6b5bd8f1d None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:35.332824 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eebb45-74ee-4336-95c8-d3c6b5bd8f1d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:35.339566 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6f4760-0b34-4689-a89e-f89cbbf27b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:35.340444 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f4760-0b34-4689-a89e-f89cbbf27b6f None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:35.340879 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f4760-0b34-4689-a89e-f89cbbf27b6f None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:35.341297 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f4760-0b34-4689-a89e-f89cbbf27b6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:35.342138 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6f4760-0b34-4689-a89e-f89cbbf27b6f None admin] RBAC: Adding query filter params (group.id=3d53537d1bfe4cf0bc60529db50c1c6a, role.id=c467f10d3dba4f2dbfd42068f42f74b6, scope.project.id=21cd4117dcd346b8bcb3e353ff8d228a) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:35.342454 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6f4760-0b34-4689-a89e-f89cbbf27b6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:35.343903 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6f4760-0b34-4689-a89e-f89cbbf27b6f None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:35.348950 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eebb45-74ee-4336-95c8-d3c6b5bd8f1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:05:35.350763 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 170/350] 10.222.0.23 () {56 vars in 1376 bytes} [Tue Jun 21 18:05:35 2022] GET /identity/v3/role_assignments?group.id=3d53537d1bfe4cf0bc60529db50c1c6a&scope.project.id=21cd4117dcd346b8bcb3e353ff8d228a&role.id=c467f10d3dba4f2dbfd42068f42f74b6 => generated 260 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:35.352183 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d621232-fd43-40ff-a981-7f023497861a None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:35.356631 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d621232-fd43-40ff-a981-7f023497861a None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:35.356988 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d621232-fd43-40ff-a981-7f023497861a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:35.357649 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-26eebb45-74ee-4336-95c8-d3c6b5bd8f1d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 18:05:35.357649 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:35.357649 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:35.357649 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:35.357649 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:35.357649 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:35.357649 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:35.357649 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:35.357649 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:35.357649 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:35.357649 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:35.357649 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:35.357649 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:05:35.357649 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:05:35.357649 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:05:35.357649 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:05:35.358545 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:05:35.358545 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:05:35.358545 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:05:35.358545 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:05:35.358545 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:35.358545 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:35.358545 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:35.358545 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:35.358545 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:35.358545 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:35.358545 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:35.358545 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:35.358545 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:35.358545 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:35.358545 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:35.358545 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:35.359391 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:35.359391 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:35.359391 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:35.359391 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:35.359391 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:05:35.359391 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:05:35.359391 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:05:35.359391 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:05:35.359391 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:05:35.359391 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:05:35.359391 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 18:05:35.359391 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:05:35.359391 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 181/351] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:05:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:35.361442 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eebb45-74ee-4336-95c8-d3c6b5bd8f1d None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:35.386812 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-9d621232-fd43-40ff-a981-7f023497861a None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:35.388140 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-9d621232-fd43-40ff-a981-7f023497861a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:35.448908 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:35.485704 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 171/352] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:05:35.486322 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:35.488555 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d621232-fd43-40ff-a981-7f023497861a None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:35.495728 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eebb45-74ee-4336-95c8-d3c6b5bd8f1d None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:35.548791 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad9f6d3-c68f-4b6f-81a7-1e629ee47601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:35.549620 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9f6d3-c68f-4b6f-81a7-1e629ee47601 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:35.550324 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9f6d3-c68f-4b6f-81a7-1e629ee47601 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:35.550324 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9f6d3-c68f-4b6f-81a7-1e629ee47601 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:35.566112 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad9f6d3-c68f-4b6f-81a7-1e629ee47601 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:05:35.578799 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-4ad9f6d3-c68f-4b6f-81a7-1e629ee47601 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:05:35.578799 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:35.578799 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:35.578799 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:35.578799 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:35.578799 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:35.578799 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:35.578799 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:35.578799 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:35.578799 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:35.578799 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:35.578799 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:35.578799 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:05:35.578799 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:05:35.578799 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:05:35.578799 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:05:35.580175 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:05:35.580175 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:05:35.580175 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:05:35.580175 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:05:35.580175 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:35.580175 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:35.580175 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:35.580175 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:35.580175 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:35.580175 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:35.580175 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:35.580175 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:35.580175 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:35.580175 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:35.580175 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:35.580175 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:35.581098 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:35.581098 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:35.581098 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:35.581098 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:35.581098 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:05:35.581098 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:05:35.581098 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:05:35.581098 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:05:35.581098 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:05:35.581098 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:05:35.581098 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:05:35.581098 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:05:35.582478 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 182/353] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:05:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:35.583003 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad9f6d3-c68f-4b6f-81a7-1e629ee47601 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:35.635699 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d621232-fd43-40ff-a981-7f023497861a None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:35.685532 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aeabc0-0b44-4428-801f-b81726406e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:35.686720 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aeabc0-0b44-4428-801f-b81726406e2b None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:35.687220 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aeabc0-0b44-4428-801f-b81726406e2b None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:35.687751 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aeabc0-0b44-4428-801f-b81726406e2b None admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:35.688804 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aeabc0-0b44-4428-801f-b81726406e2b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:35.689323 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aeabc0-0b44-4428-801f-b81726406e2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:35.691577 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aeabc0-0b44-4428-801f-b81726406e2b None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:35.701785 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 172/354] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jun 21 18:05:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:35.703121 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aeabc0-0b44-4428-801f-b81726406e2b None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:35.722116 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad9f6d3-c68f-4b6f-81a7-1e629ee47601 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:35.755379 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:35.779136 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e30c6e7-7829-4f07-ba24-dc8ac4dcaa83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:35.779838 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e30c6e7-7829-4f07-ba24-dc8ac4dcaa83 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:35.780114 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e30c6e7-7829-4f07-ba24-dc8ac4dcaa83 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:35.780466 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e30c6e7-7829-4f07-ba24-dc8ac4dcaa83 None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:35.781564 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e30c6e7-7829-4f07-ba24-dc8ac4dcaa83 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:35.781878 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e30c6e7-7829-4f07-ba24-dc8ac4dcaa83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:35.783546 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e30c6e7-7829-4f07-ba24-dc8ac4dcaa83 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:35.797774 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 183/355] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jun 21 18:05:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:35.799031 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e30c6e7-7829-4f07-ba24-dc8ac4dcaa83 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:35.825983 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:35.836270 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aeabc0-0b44-4428-801f-b81726406e2b None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:35.885721 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca06e43f-9e9e-41ff-b70d-ce484e8f26e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:35.886623 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca06e43f-9e9e-41ff-b70d-ce484e8f26e3 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:35.888863 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca06e43f-9e9e-41ff-b70d-ce484e8f26e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:35.888863 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca06e43f-9e9e-41ff-b70d-ce484e8f26e3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:35.904847 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca06e43f-9e9e-41ff-b70d-ce484e8f26e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:05:35.913879 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-ca06e43f-9e9e-41ff-b70d-ce484e8f26e3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:05:35.913879 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:35.913879 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:35.913879 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:35.913879 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:35.913879 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:35.913879 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:35.913879 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:35.913879 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:35.913879 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:35.913879 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:35.913879 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:35.913879 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:05:35.913879 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:05:35.913879 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:05:35.913879 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:05:35.915030 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:05:35.915030 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:05:35.915030 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:05:35.915030 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:05:35.915030 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:35.915030 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:35.915030 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:35.915030 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:35.915030 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:35.915030 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:35.915030 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:35.915030 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:35.915030 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:35.915030 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:35.915030 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:35.915030 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:35.916892 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:35.916892 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:35.916892 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:35.916892 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:35.916892 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:05:35.916892 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:05:35.916892 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:05:35.916892 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:05:35.916892 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:05:35.916892 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:05:35.916892 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:05:35.916892 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:05:35.918063 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 173/356] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jun 21 18:05:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:35.919097 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca06e43f-9e9e-41ff-b70d-ce484e8f26e3 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:35.947500 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e30c6e7-7829-4f07-ba24-dc8ac4dcaa83 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:36.000788 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc82b9-ce64-4924-b9fa-b385049984a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:36.001480 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc82b9-ce64-4924-b9fa-b385049984a0 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:36.001761 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc82b9-ce64-4924-b9fa-b385049984a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:36.002039 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc82b9-ce64-4924-b9fa-b385049984a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:36.002803 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dc82b9-ce64-4924-b9fa-b385049984a0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:36.003111 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dc82b9-ce64-4924-b9fa-b385049984a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:36.006967 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dc82b9-ce64-4924-b9fa-b385049984a0 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:36.015965 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 184/357] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jun 21 18:05:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:36.016887 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc82b9-ce64-4924-b9fa-b385049984a0 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:36.053607 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:36.061433 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca06e43f-9e9e-41ff-b70d-ce484e8f26e3 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:36.116667 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069e2274-0607-4c6c-adcb-70b9e60d9fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:36.117427 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e2274-0607-4c6c-adcb-70b9e60d9fac None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:36.117717 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e2274-0607-4c6c-adcb-70b9e60d9fac None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:36.117998 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e2274-0607-4c6c-adcb-70b9e60d9fac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:36.125218 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:36.153896 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc82b9-ce64-4924-b9fa-b385049984a0 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:36.197640 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad695de7-16f8-40c0-aa02-d22be5145df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:36.198663 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad695de7-16f8-40c0-aa02-d22be5145df0 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:36.198951 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad695de7-16f8-40c0-aa02-d22be5145df0 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:36.199230 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad695de7-16f8-40c0-aa02-d22be5145df0 None admin] PATH_INFO: `/v3/users/6a2dc128e93842c894d499be281ed626` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:36.200543 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad695de7-16f8-40c0-aa02-d22be5145df0 None admin] RBAC: Authorizing `identity:get_user(user_id=6a2dc128e93842c894d499be281ed626)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:36.201988 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad695de7-16f8-40c0-aa02-d22be5145df0 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:36.204170 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 185/358] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jun 21 18:05:36 2022] GET /identity/v3/users/6a2dc128e93842c894d499be281ed626 => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:36.205458 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09d732a-d42b-4dc5-bc02-d7783ffa2526 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:36.205643 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069e2274-0607-4c6c-adcb-70b9e60d9fac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:36.205754 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09d732a-d42b-4dc5-bc02-d7783ffa2526 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:36.206011 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09d732a-d42b-4dc5-bc02-d7783ffa2526 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:36.207084 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069e2274-0607-4c6c-adcb-70b9e60d9fac None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:36.214374 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 186/359] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:36 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:05:36.215542 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09d732a-d42b-4dc5-bc02-d7783ffa2526 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:36.336271 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:36.340172 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 174/360] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:05:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:05:36.341510 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069e2274-0607-4c6c-adcb-70b9e60d9fac None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:36.369727 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09d732a-d42b-4dc5-bc02-d7783ffa2526 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:36.417470 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad11a9d2-192a-408d-b313-091ca1a814f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:36.418156 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad11a9d2-192a-408d-b313-091ca1a814f5 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:36.418428 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad11a9d2-192a-408d-b313-091ca1a814f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:36.418790 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad11a9d2-192a-408d-b313-091ca1a814f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:36.419464 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad11a9d2-192a-408d-b313-091ca1a814f5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:36.419773 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad11a9d2-192a-408d-b313-091ca1a814f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:36.421180 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad11a9d2-192a-408d-b313-091ca1a814f5 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:36.434904 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 187/361] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jun 21 18:05:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:36.434904 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad11a9d2-192a-408d-b313-091ca1a814f5 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:36.442055 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:36.483370 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069e2274-0607-4c6c-adcb-70b9e60d9fac None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:36.524873 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75be22f-a949-4949-8a3c-49b3b3fccf70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:36.526058 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75be22f-a949-4949-8a3c-49b3b3fccf70 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:36.526513 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75be22f-a949-4949-8a3c-49b3b3fccf70 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:36.526686 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75be22f-a949-4949-8a3c-49b3b3fccf70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:36.557841 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad11a9d2-192a-408d-b313-091ca1a814f5 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:36.599904 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923d1b20-c482-4b46-ae12-0f19d22afa7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:36.600889 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d1b20-c482-4b46-ae12-0f19d22afa7e None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:36.601332 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d1b20-c482-4b46-ae12-0f19d22afa7e None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:36.601836 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d1b20-c482-4b46-ae12-0f19d22afa7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:36.602953 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923d1b20-c482-4b46-ae12-0f19d22afa7e None admin] RBAC: Adding query filter params (role.id=55af05e4b36e44d282933b6a18958580, user.id=6a2dc128e93842c894d499be281ed626, scope.system=all) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:36.604030 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923d1b20-c482-4b46-ae12-0f19d22afa7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:36.605774 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923d1b20-c482-4b46-ae12-0f19d22afa7e None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:36.616725 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:36.625873 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 188/362] 10.222.0.23 () {56 vars in 1308 bytes} [Tue Jun 21 18:05:36 2022] GET /identity/v3/role_assignments?user.id=6a2dc128e93842c894d499be281ed626&scope.system=all&role.id=55af05e4b36e44d282933b6a18958580 => generated 510 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:36.625873 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d698a597-4904-409c-af12-6c765cb41d2f None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:36.625873 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d698a597-4904-409c-af12-6c765cb41d2f None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:36.625873 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d698a597-4904-409c-af12-6c765cb41d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:36.626368 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75be22f-a949-4949-8a3c-49b3b3fccf70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:36.626368 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75be22f-a949-4949-8a3c-49b3b3fccf70 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:36.643193 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-d698a597-4904-409c-af12-6c765cb41d2f None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:36.644868 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-d698a597-4904-409c-af12-6c765cb41d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:36.722160 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:36.731683 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 189/363] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:05:36.736765 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d698a597-4904-409c-af12-6c765cb41d2f None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:36.771302 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 175/364] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:05:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:05:36.772923 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75be22f-a949-4949-8a3c-49b3b3fccf70 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:36.902983 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d698a597-4904-409c-af12-6c765cb41d2f None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:36.911591 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:36.920570 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75be22f-a949-4949-8a3c-49b3b3fccf70 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:36.952638 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e212ce8a-8756-4931-8034-aa89892b5507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:36.953521 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212ce8a-8756-4931-8034-aa89892b5507 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:36.953805 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212ce8a-8756-4931-8034-aa89892b5507 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:36.954077 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212ce8a-8756-4931-8034-aa89892b5507 None admin] PATH_INFO: `/v3/projects/21cd4117dcd346b8bcb3e353ff8d228a` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:36.955301 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e212ce8a-8756-4931-8034-aa89892b5507 None admin] RBAC: Authorizing `identity:get_project(project_id=21cd4117dcd346b8bcb3e353ff8d228a)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:36.956834 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e212ce8a-8756-4931-8034-aa89892b5507 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:36.958530 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 190/365] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jun 21 18:05:36 2022] GET /identity/v3/projects/21cd4117dcd346b8bcb3e353ff8d228a => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:36.959385 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e212ce8a-8756-4931-8034-aa89892b5507 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:36.971709 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c67c4-0da4-419e-a6ef-5c8420e47178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:36.972443 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c67c4-0da4-419e-a6ef-5c8420e47178 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:36.972827 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c67c4-0da4-419e-a6ef-5c8420e47178 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:36.973115 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c67c4-0da4-419e-a6ef-5c8420e47178 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:37.039021 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:37.075606 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3c67c4-0da4-419e-a6ef-5c8420e47178 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:37.079143 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3c67c4-0da4-419e-a6ef-5c8420e47178 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:37.096364 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e212ce8a-8756-4931-8034-aa89892b5507 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:37.143258 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df5e9bc-691f-426a-a240-2ca09f7e7358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:37.144097 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5e9bc-691f-426a-a240-2ca09f7e7358 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:37.144343 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5e9bc-691f-426a-a240-2ca09f7e7358 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:37.144650 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5e9bc-691f-426a-a240-2ca09f7e7358 None admin] PATH_INFO: `/v3/projects/53e988a4b11648fcb59e0748d5dbf37b` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:37.146707 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df5e9bc-691f-426a-a240-2ca09f7e7358 None admin] RBAC: Authorizing `identity:get_project(project_id=53e988a4b11648fcb59e0748d5dbf37b)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:37.148172 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df5e9bc-691f-426a-a240-2ca09f7e7358 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:37.151015 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 191/366] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jun 21 18:05:36 2022] GET /identity/v3/projects/53e988a4b11648fcb59e0748d5dbf37b => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:37.151702 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df5e9bc-691f-426a-a240-2ca09f7e7358 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:37.210335 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:37.222307 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 176/367] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:05:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:05:37.228958 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c67c4-0da4-419e-a6ef-5c8420e47178 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:37.287227 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df5e9bc-691f-426a-a240-2ca09f7e7358 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:37.322355 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:37.332570 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374d9e88-d805-4874-83d4-cfd771c08b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:37.333387 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d9e88-d805-4874-83d4-cfd771c08b44 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:37.335834 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d9e88-d805-4874-83d4-cfd771c08b44 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:37.335834 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d9e88-d805-4874-83d4-cfd771c08b44 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:37.355121 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374d9e88-d805-4874-83d4-cfd771c08b44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:05:37.363337 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-374d9e88-d805-4874-83d4-cfd771c08b44 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:05:37.363337 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:37.363337 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:37.363337 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:37.363337 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:37.363337 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:37.363337 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:37.363337 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:37.363337 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:37.363337 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:37.363337 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:37.363337 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:37.363337 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:05:37.363337 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:05:37.363337 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:05:37.363337 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:05:37.364245 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:05:37.364245 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:05:37.364245 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:05:37.364245 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:05:37.364245 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:37.364245 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:37.364245 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:37.364245 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:37.364245 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:37.364245 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:37.364245 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:37.364245 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:37.364245 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:37.364245 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:37.364245 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:37.364245 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:37.365118 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:37.365118 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:37.365118 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:37.365118 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:37.365118 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:05:37.365118 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:05:37.365118 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:05:37.365118 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:05:37.365118 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:05:37.365118 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:05:37.365118 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:05:37.365118 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:05:37.365748 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c67c4-0da4-419e-a6ef-5c8420e47178 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:37.367616 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 192/368] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jun 21 18:05:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:37.368602 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374d9e88-d805-4874-83d4-cfd771c08b44 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:37.404943 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f63fdca-c926-465a-afc9-bff0c2d8c9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:37.405817 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f63fdca-c926-465a-afc9-bff0c2d8c9de None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:37.406108 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f63fdca-c926-465a-afc9-bff0c2d8c9de None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:37.406396 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f63fdca-c926-465a-afc9-bff0c2d8c9de None admin] PATH_INFO: `/v3/groups/3d53537d1bfe4cf0bc60529db50c1c6a` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:37.409156 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f63fdca-c926-465a-afc9-bff0c2d8c9de None admin] RBAC: Authorizing `identity:get_group(group_id=3d53537d1bfe4cf0bc60529db50c1c6a)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:37.410753 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f63fdca-c926-465a-afc9-bff0c2d8c9de None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:37.412434 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 177/369] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Jun 21 18:05:37 2022] GET /identity/v3/groups/3d53537d1bfe4cf0bc60529db50c1c6a => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:37.414322 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f63fdca-c926-465a-afc9-bff0c2d8c9de None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:37.500787 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374d9e88-d805-4874-83d4-cfd771c08b44 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:37.504593 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:37.543477 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d2f1ff-1267-4cb1-9c62-903269c92754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:37.544361 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d2f1ff-1267-4cb1-9c62-903269c92754 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:37.544717 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d2f1ff-1267-4cb1-9c62-903269c92754 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:37.545199 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d2f1ff-1267-4cb1-9c62-903269c92754 None admin] PATH_INFO: `/v3/groups/3d53537d1bfe4cf0bc60529db50c1c6a` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:37.546496 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d2f1ff-1267-4cb1-9c62-903269c92754 None admin] RBAC: Authorizing `identity:get_group(group_id=3d53537d1bfe4cf0bc60529db50c1c6a)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:37.548668 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d2f1ff-1267-4cb1-9c62-903269c92754 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:37.550384 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 193/370] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Jun 21 18:05:37 2022] GET /identity/v3/groups/3d53537d1bfe4cf0bc60529db50c1c6a => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:37.551493 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d2f1ff-1267-4cb1-9c62-903269c92754 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:37.557305 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f63fdca-c926-465a-afc9-bff0c2d8c9de None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:37.601743 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7531f3-6283-4fb9-aa16-460757ae596f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:37.602653 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7531f3-6283-4fb9-aa16-460757ae596f None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:37.602933 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7531f3-6283-4fb9-aa16-460757ae596f None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:37.603207 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7531f3-6283-4fb9-aa16-460757ae596f None admin] PATH_INFO: `/v3/projects/53e988a4b11648fcb59e0748d5dbf37b` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:37.604961 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7531f3-6283-4fb9-aa16-460757ae596f None admin] RBAC: Authorizing `identity:get_project(project_id=53e988a4b11648fcb59e0748d5dbf37b)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:37.606849 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7531f3-6283-4fb9-aa16-460757ae596f None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:37.608219 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 178/371] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jun 21 18:05:37 2022] GET /identity/v3/projects/53e988a4b11648fcb59e0748d5dbf37b => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:37.609775 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7531f3-6283-4fb9-aa16-460757ae596f None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:37.633487 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:37.694802 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d2f1ff-1267-4cb1-9c62-903269c92754 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:37.733221 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7531f3-6283-4fb9-aa16-460757ae596f None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:37.738308 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdf68e6-e567-48b4-b01e-e19d661f1c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:37.738990 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdf68e6-e567-48b4-b01e-e19d661f1c8b None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:37.739276 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdf68e6-e567-48b4-b01e-e19d661f1c8b None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:37.739551 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdf68e6-e567-48b4-b01e-e19d661f1c8b None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:37.740307 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdf68e6-e567-48b4-b01e-e19d661f1c8b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:37.741047 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdf68e6-e567-48b4-b01e-e19d661f1c8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:37.742420 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdf68e6-e567-48b4-b01e-e19d661f1c8b None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:37.753810 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 194/372] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:05:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:37.754667 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdf68e6-e567-48b4-b01e-e19d661f1c8b None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:37.777886 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f176f57-b27c-444e-b436-24e952e31ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:37.778705 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f176f57-b27c-444e-b436-24e952e31ed4 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:37.779378 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f176f57-b27c-444e-b436-24e952e31ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:37.779378 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f176f57-b27c-444e-b436-24e952e31ed4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:37.780218 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f176f57-b27c-444e-b436-24e952e31ed4 None admin] RBAC: Adding query filter params (group.id=3d53537d1bfe4cf0bc60529db50c1c6a, role.id=482dcc9ba0fd400cad2d193e3e895d4c, scope.project.id=21cd4117dcd346b8bcb3e353ff8d228a) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:37.780623 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f176f57-b27c-444e-b436-24e952e31ed4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:37.783028 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f176f57-b27c-444e-b436-24e952e31ed4 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:37.784002 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:37.789192 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 179/373] 10.222.0.23 () {56 vars in 1376 bytes} [Tue Jun 21 18:05:37 2022] GET /identity/v3/role_assignments?group.id=3d53537d1bfe4cf0bc60529db50c1c6a&scope.project.id=21cd4117dcd346b8bcb3e353ff8d228a&role.id=482dcc9ba0fd400cad2d193e3e895d4c => generated 260 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:37.790080 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f176f57-b27c-444e-b436-24e952e31ed4 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:37.902136 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdf68e6-e567-48b4-b01e-e19d661f1c8b None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:37.931861 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:37.952032 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec2c175-9777-4a23-9c5b-653e5c41a36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:37.952272 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec2c175-9777-4a23-9c5b-653e5c41a36c None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:37.952438 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec2c175-9777-4a23-9c5b-653e5c41a36c None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:37.952681 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec2c175-9777-4a23-9c5b-653e5c41a36c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:37.953542 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec2c175-9777-4a23-9c5b-653e5c41a36c None admin] RBAC: Adding query filter params (group.id=3d53537d1bfe4cf0bc60529db50c1c6a, role.id=c467f10d3dba4f2dbfd42068f42f74b6, scope.project.id=53e988a4b11648fcb59e0748d5dbf37b) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:37.953701 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f176f57-b27c-444e-b436-24e952e31ed4 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:37.953882 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec2c175-9777-4a23-9c5b-653e5c41a36c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:37.955333 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec2c175-9777-4a23-9c5b-653e5c41a36c None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:37.961490 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 195/374] 10.222.0.23 () {56 vars in 1376 bytes} [Tue Jun 21 18:05:37 2022] GET /identity/v3/role_assignments?group.id=3d53537d1bfe4cf0bc60529db50c1c6a&scope.project.id=53e988a4b11648fcb59e0748d5dbf37b&role.id=c467f10d3dba4f2dbfd42068f42f74b6 => generated 260 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:37.969907 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe96f40-fb31-4fbe-9565-4fcad2113005 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:37.970145 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe96f40-fb31-4fbe-9565-4fcad2113005 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:37.970439 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe96f40-fb31-4fbe-9565-4fcad2113005 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:37.973042 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 196/375] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:37 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:05:37.973184 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe96f40-fb31-4fbe-9565-4fcad2113005 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:38.017620 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325dbf24-ce86-4dc0-a26b-99d17c294bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:38.019201 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325dbf24-ce86-4dc0-a26b-99d17c294bf5 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:38.019493 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325dbf24-ce86-4dc0-a26b-99d17c294bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:38.019777 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325dbf24-ce86-4dc0-a26b-99d17c294bf5 None admin] PATH_INFO: `/v3/groups/3d53537d1bfe4cf0bc60529db50c1c6a` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:38.021847 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325dbf24-ce86-4dc0-a26b-99d17c294bf5 None admin] RBAC: Authorizing `identity:get_group(group_id=3d53537d1bfe4cf0bc60529db50c1c6a)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:38.023651 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325dbf24-ce86-4dc0-a26b-99d17c294bf5 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:38.025493 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 180/376] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Jun 21 18:05:37 2022] GET /identity/v3/groups/3d53537d1bfe4cf0bc60529db50c1c6a => generated 226 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:38.026830 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d949fab-e74d-4201-8d34-055a8607ed3b None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:38.027174 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d949fab-e74d-4201-8d34-055a8607ed3b None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:38.027458 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d949fab-e74d-4201-8d34-055a8607ed3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:38.049535 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-6d949fab-e74d-4201-8d34-055a8607ed3b None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:38.051246 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-6d949fab-e74d-4201-8d34-055a8607ed3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:38.118130 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe96f40-fb31-4fbe-9565-4fcad2113005 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:38.137080 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:38.146422 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 181/377] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:05:38.148276 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d949fab-e74d-4201-8d34-055a8607ed3b None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:38.159685 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db0bc58-cdaf-4242-a4cc-f47869d059f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:38.160490 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db0bc58-cdaf-4242-a4cc-f47869d059f0 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:38.160875 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db0bc58-cdaf-4242-a4cc-f47869d059f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:38.161226 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db0bc58-cdaf-4242-a4cc-f47869d059f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:38.226230 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:38.240640 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db0bc58-cdaf-4242-a4cc-f47869d059f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:38.242224 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db0bc58-cdaf-4242-a4cc-f47869d059f0 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:38.277571 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d949fab-e74d-4201-8d34-055a8607ed3b None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:38.315613 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febdb094-2cf2-4f08-8a01-bd1559ece9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:38.316617 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febdb094-2cf2-4f08-8a01-bd1559ece9b7 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:38.317075 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febdb094-2cf2-4f08-8a01-bd1559ece9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:38.317542 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febdb094-2cf2-4f08-8a01-bd1559ece9b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:38.318567 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febdb094-2cf2-4f08-8a01-bd1559ece9b7 None admin] RBAC: Adding query filter params (group.id=3d53537d1bfe4cf0bc60529db50c1c6a, role.id=482dcc9ba0fd400cad2d193e3e895d4c, scope.project.id=53e988a4b11648fcb59e0748d5dbf37b) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:38.319052 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febdb094-2cf2-4f08-8a01-bd1559ece9b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:38.320718 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febdb094-2cf2-4f08-8a01-bd1559ece9b7 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:38.328369 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 182/378] 10.222.0.23 () {56 vars in 1376 bytes} [Tue Jun 21 18:05:38 2022] GET /identity/v3/role_assignments?group.id=3d53537d1bfe4cf0bc60529db50c1c6a&scope.project.id=53e988a4b11648fcb59e0748d5dbf37b&role.id=482dcc9ba0fd400cad2d193e3e895d4c => generated 260 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:38.329730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febdb094-2cf2-4f08-8a01-bd1559ece9b7 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:38.357943 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 197/379] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:05:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:05:38.370735 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db0bc58-cdaf-4242-a4cc-f47869d059f0 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:38.403895 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:38.452723 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febdb094-2cf2-4f08-8a01-bd1559ece9b7 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:38.496458 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64807513-486c-4186-a270-109347cd3da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:38.497631 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64807513-486c-4186-a270-109347cd3da5 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:38.498122 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64807513-486c-4186-a270-109347cd3da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:38.498632 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64807513-486c-4186-a270-109347cd3da5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:38.517808 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64807513-486c-4186-a270-109347cd3da5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:05:38.526148 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-64807513-486c-4186-a270-109347cd3da5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:05:38.526148 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:38.526148 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:38.526148 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:38.526148 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:38.526148 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:38.526148 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:38.526148 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:38.526148 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:38.526148 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:38.526148 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:38.526148 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:38.526148 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:05:38.526148 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:05:38.526148 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:05:38.526148 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:05:38.527098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:05:38.527098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:05:38.527098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:05:38.527098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:05:38.527098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:38.527098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:38.527098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:38.527098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:38.527098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:38.527098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:38.527098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:38.527098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:38.527098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:38.527098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:38.527098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:38.527098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:38.527993 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:38.527993 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:38.527993 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:38.527993 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:38.527993 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:05:38.527993 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:05:38.527993 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:05:38.527993 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:05:38.527993 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:05:38.527993 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:05:38.527993 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:05:38.527993 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:05:38.532370 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 183/380] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:05:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:38.533087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:38.533665 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64807513-486c-4186-a270-109347cd3da5 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:38.554555 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db0bc58-cdaf-4242-a4cc-f47869d059f0 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:38.600918 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d29067-5c90-4861-80b6-4eb46aa2b8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:38.602318 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d29067-5c90-4861-80b6-4eb46aa2b8f7 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:38.602847 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d29067-5c90-4861-80b6-4eb46aa2b8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:38.603960 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d29067-5c90-4861-80b6-4eb46aa2b8f7 None admin] PATH_INFO: `/v3/projects/cecad7a8a62644cabfc30fc28ce80d90` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:38.606317 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d29067-5c90-4861-80b6-4eb46aa2b8f7 None admin] RBAC: Authorizing `identity:get_project(project_id=cecad7a8a62644cabfc30fc28ce80d90)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:38.608356 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d29067-5c90-4861-80b6-4eb46aa2b8f7 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:38.610633 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 198/381] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jun 21 18:05:38 2022] GET /identity/v3/projects/cecad7a8a62644cabfc30fc28ce80d90 => generated 344 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:38.615755 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d29067-5c90-4861-80b6-4eb46aa2b8f7 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:38.699214 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64807513-486c-4186-a270-109347cd3da5 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:38.757490 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:38.766854 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d29067-5c90-4861-80b6-4eb46aa2b8f7 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:38.805689 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b430d713-0b4c-48b5-bef2-efe5f0887bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:38.806732 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b430d713-0b4c-48b5-bef2-efe5f0887bdc None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:38.807244 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b430d713-0b4c-48b5-bef2-efe5f0887bdc None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:38.807244 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b430d713-0b4c-48b5-bef2-efe5f0887bdc None admin] PATH_INFO: `/v3/groups/efb3e72636cc4d62818b957d52674420` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:38.812659 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b430d713-0b4c-48b5-bef2-efe5f0887bdc None admin] RBAC: Authorizing `identity:get_group(group_id=efb3e72636cc4d62818b957d52674420)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:38.814133 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b430d713-0b4c-48b5-bef2-efe5f0887bdc None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:38.816110 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 199/382] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Jun 21 18:05:38 2022] GET /identity/v3/groups/efb3e72636cc4d62818b957d52674420 => generated 229 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:38.817603 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5954df88-17d0-404a-97b8-6befd30ae4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:38.818354 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954df88-17d0-404a-97b8-6befd30ae4cd None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:38.818707 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954df88-17d0-404a-97b8-6befd30ae4cd None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:38.819049 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954df88-17d0-404a-97b8-6befd30ae4cd None admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:38.819969 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5954df88-17d0-404a-97b8-6befd30ae4cd None admin] RBAC: Adding query filter params (name=member) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:38.820346 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5954df88-17d0-404a-97b8-6befd30ae4cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:38.821945 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b430d713-0b4c-48b5-bef2-efe5f0887bdc None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:38.831803 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5954df88-17d0-404a-97b8-6befd30ae4cd None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:38.849899 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 184/383] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jun 21 18:05:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:05:38.851211 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5954df88-17d0-404a-97b8-6befd30ae4cd None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:38.856420 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:38.983465 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5954df88-17d0-404a-97b8-6befd30ae4cd None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:39.044871 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b430d713-0b4c-48b5-bef2-efe5f0887bdc None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:39.054505 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff99f3-319c-4200-8b87-218e624e2c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:39.055384 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff99f3-319c-4200-8b87-218e624e2c56 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:39.055665 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff99f3-319c-4200-8b87-218e624e2c56 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:39.055935 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff99f3-319c-4200-8b87-218e624e2c56 None admin] PATH_INFO: `/v3/groups/3d53537d1bfe4cf0bc60529db50c1c6a` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:39.059261 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cff99f3-319c-4200-8b87-218e624e2c56 None admin] RBAC: Authorizing `identity:get_group(group_id=3d53537d1bfe4cf0bc60529db50c1c6a)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:39.060773 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cff99f3-319c-4200-8b87-218e624e2c56 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:39.062556 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 185/384] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Jun 21 18:05:38 2022] GET /identity/v3/groups/3d53537d1bfe4cf0bc60529db50c1c6a => generated 226 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:39.068115 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff99f3-319c-4200-8b87-218e624e2c56 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:39.112236 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0a5fba-f6f9-45d7-83c1-426375074929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:39.112980 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a5fba-f6f9-45d7-83c1-426375074929 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:39.113270 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a5fba-f6f9-45d7-83c1-426375074929 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:39.113548 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a5fba-f6f9-45d7-83c1-426375074929 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:39.114390 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0a5fba-f6f9-45d7-83c1-426375074929 None admin] RBAC: Adding query filter params (group.id=efb3e72636cc4d62818b957d52674420, role.id=745ed9065a8143e384ebbd694b29635c, scope.project.id=cecad7a8a62644cabfc30fc28ce80d90) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:39.114702 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0a5fba-f6f9-45d7-83c1-426375074929 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:39.116133 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0a5fba-f6f9-45d7-83c1-426375074929 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:39.129409 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 200/385] 10.222.0.23 () {56 vars in 1376 bytes} [Tue Jun 21 18:05:38 2022] GET /identity/v3/role_assignments?group.id=efb3e72636cc4d62818b957d52674420&scope.project.id=cecad7a8a62644cabfc30fc28ce80d90&role.id=745ed9065a8143e384ebbd694b29635c => generated 260 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:05:39.164800 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:39.265229 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff99f3-319c-4200-8b87-218e624e2c56 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:39.313744 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6df6e-0876-423f-a639-0e2e88bfddbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:39.315060 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6df6e-0876-423f-a639-0e2e88bfddbf None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:39.315538 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6df6e-0876-423f-a639-0e2e88bfddbf None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:39.316004 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6df6e-0876-423f-a639-0e2e88bfddbf None admin] PATH_INFO: `/v3/projects/21cd4117dcd346b8bcb3e353ff8d228a` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:39.317414 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b6df6e-0876-423f-a639-0e2e88bfddbf None admin] RBAC: Authorizing `identity:get_project(project_id=21cd4117dcd346b8bcb3e353ff8d228a)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:39.319044 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b6df6e-0876-423f-a639-0e2e88bfddbf None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:39.321277 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 186/386] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jun 21 18:05:39 2022] GET /identity/v3/projects/21cd4117dcd346b8bcb3e353ff8d228a => generated 298 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:05:39.327233 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0a5fba-f6f9-45d7-83c1-426375074929 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:39.438648 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:39.465078 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0a5fba-f6f9-45d7-83c1-426375074929 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:39.522090 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef031cc8-e028-41c0-be7f-d7e65ded7e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:39.523368 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef031cc8-e028-41c0-be7f-d7e65ded7e2c None admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:39.523853 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef031cc8-e028-41c0-be7f-d7e65ded7e2c None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:39.524319 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef031cc8-e028-41c0-be7f-d7e65ded7e2c None admin] PATH_INFO: `/v3/projects/21cd4117dcd346b8bcb3e353ff8d228a/groups/3d53537d1bfe4cf0bc60529db50c1c6a/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:39.529726 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef031cc8-e028-41c0-be7f-d7e65ded7e2c None admin] RBAC: Authorizing `identity:create_grant(project_id=21cd4117dcd346b8bcb3e353ff8d228a, group_id=3d53537d1bfe4cf0bc60529db50c1c6a, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:39.531606 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef031cc8-e028-41c0-be7f-d7e65ded7e2c None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:39.623710 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 201/387] 10.222.0.23 () {58 vars in 1442 bytes} [Tue Jun 21 18:05:39 2022] PUT /identity/v3/projects/21cd4117dcd346b8bcb3e353ff8d228a/groups/3d53537d1bfe4cf0bc60529db50c1c6a/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:05:40.003221 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1de067-6693-4906-93b7-ce706d9db982 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:40.003754 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1de067-6693-4906-93b7-ce706d9db982 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:40.003833 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1de067-6693-4906-93b7-ce706d9db982 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:40.005469 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 187/388] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:05:40.011765 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c935f-2529-4add-830d-dc519dbdd675 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:40.012058 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c935f-2529-4add-830d-dc519dbdd675 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:40.012343 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c935f-2529-4add-830d-dc519dbdd675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:40.030625 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-613c935f-2529-4add-830d-dc519dbdd675 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:40.031883 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-613c935f-2529-4add-830d-dc519dbdd675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:40.105088 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 202/389] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:05:40.132871 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1de067-6693-4906-93b7-ce706d9db982 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:40.278133 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1de067-6693-4906-93b7-ce706d9db982 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:40.299477 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:40.325174 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ec78a8-057e-4acd-aa26-cdb1c0e0c19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:40.326017 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec78a8-057e-4acd-aa26-cdb1c0e0c19c None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:40.326304 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec78a8-057e-4acd-aa26-cdb1c0e0c19c None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:40.326582 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec78a8-057e-4acd-aa26-cdb1c0e0c19c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:40.341407 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ec78a8-057e-4acd-aa26-cdb1c0e0c19c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:05:40.349817 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-43ec78a8-057e-4acd-aa26-cdb1c0e0c19c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:05:40.349817 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:40.349817 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:40.349817 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:40.349817 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:40.349817 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:40.349817 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:40.349817 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:40.349817 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:40.349817 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:40.349817 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:40.349817 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:40.349817 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:05:40.349817 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:05:40.349817 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:05:40.349817 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:05:40.351837 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:05:40.351837 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:05:40.351837 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:05:40.351837 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:05:40.351837 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:40.351837 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:40.351837 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:40.351837 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:40.351837 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:40.351837 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:40.351837 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:40.351837 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:40.351837 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:40.351837 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:40.351837 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:40.351837 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:40.354792 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:40.354792 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:40.354792 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:40.354792 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:40.354792 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:05:40.354792 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:05:40.354792 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:05:40.354792 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:05:40.354792 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:05:40.354792 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:05:40.354792 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:05:40.354792 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:05:40.354792 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 188/390] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jun 21 18:05:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:40.360431 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613c935f-2529-4add-830d-dc519dbdd675 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:40.436331 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:40.463963 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832e690-4e71-43d6-89d9-a2ef1991b9f1 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:40.464708 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832e690-4e71-43d6-89d9-a2ef1991b9f1 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:40.465221 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832e690-4e71-43d6-89d9-a2ef1991b9f1 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:40.467204 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 189/391] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:05:40.471098 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613c935f-2529-4add-830d-dc519dbdd675 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:40.473003 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87095f05-83be-48f4-86d9-bf55e40623f0 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:40.473119 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87095f05-83be-48f4-86d9-bf55e40623f0 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:40.473391 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87095f05-83be-48f4-86d9-bf55e40623f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:40.500845 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-87095f05-83be-48f4-86d9-bf55e40623f0 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:40.502313 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-87095f05-83be-48f4-86d9-bf55e40623f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:40.522029 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f95c138-1c5a-4a46-9596-8c5ab265f509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:40.522715 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f95c138-1c5a-4a46-9596-8c5ab265f509 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:40.523001 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f95c138-1c5a-4a46-9596-8c5ab265f509 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:40.523282 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f95c138-1c5a-4a46-9596-8c5ab265f509 None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:40.524046 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f95c138-1c5a-4a46-9596-8c5ab265f509 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:40.524352 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f95c138-1c5a-4a46-9596-8c5ab265f509 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:40.525977 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f95c138-1c5a-4a46-9596-8c5ab265f509 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:40.540252 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 203/392] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jun 21 18:05:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:40.546535 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f95c138-1c5a-4a46-9596-8c5ab265f509 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:40.616151 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 190/393] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:05:40.653894 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87095f05-83be-48f4-86d9-bf55e40623f0 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:40.681157 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f95c138-1c5a-4a46-9596-8c5ab265f509 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:40.747308 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:40.759925 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94107bd2-b554-4839-9f58-bfbf8f7f1760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:40.760874 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94107bd2-b554-4839-9f58-bfbf8f7f1760 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:40.761159 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94107bd2-b554-4839-9f58-bfbf8f7f1760 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:40.761441 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94107bd2-b554-4839-9f58-bfbf8f7f1760 None admin] PATH_INFO: `/v3/groups/3d53537d1bfe4cf0bc60529db50c1c6a` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:40.762783 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94107bd2-b554-4839-9f58-bfbf8f7f1760 None admin] RBAC: Authorizing `identity:get_group(group_id=3d53537d1bfe4cf0bc60529db50c1c6a)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:40.764791 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94107bd2-b554-4839-9f58-bfbf8f7f1760 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:40.766703 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 204/394] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Jun 21 18:05:40 2022] GET /identity/v3/groups/3d53537d1bfe4cf0bc60529db50c1c6a => generated 226 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:40.777410 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94107bd2-b554-4839-9f58-bfbf8f7f1760 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:40.790361 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87095f05-83be-48f4-86d9-bf55e40623f0 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:40.801076 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:40.828875 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e90befa-9ac2-495a-a420-d443bb461faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:40.829955 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e90befa-9ac2-495a-a420-d443bb461faf None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:40.830381 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e90befa-9ac2-495a-a420-d443bb461faf None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:40.830802 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e90befa-9ac2-495a-a420-d443bb461faf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:40.848421 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e90befa-9ac2-495a-a420-d443bb461faf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:05:40.859810 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-5e90befa-9ac2-495a-a420-d443bb461faf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:05:40.859810 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:40.859810 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:40.859810 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:40.859810 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:40.859810 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:40.859810 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:40.859810 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:40.859810 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:40.859810 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:40.859810 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:40.859810 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:40.859810 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:05:40.859810 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:05:40.859810 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:05:40.859810 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:05:40.860893 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:05:40.860893 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:05:40.860893 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:05:40.860893 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:05:40.860893 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:40.860893 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:40.860893 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:40.860893 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:40.860893 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:40.860893 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:40.860893 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:40.860893 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:40.860893 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:40.860893 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:40.860893 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:40.860893 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:40.861724 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:40.861724 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:40.861724 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:40.861724 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:40.861724 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:05:40.861724 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:05:40.861724 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:05:40.861724 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:05:40.861724 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:05:40.861724 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:05:40.861724 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:05:40.861724 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:05:40.862386 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 191/395] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:05:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:05:40.863858 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2257af01-b49f-463b-ba0a-8e90c3709e4e None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:40.864255 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2257af01-b49f-463b-ba0a-8e90c3709e4e None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:40.864643 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2257af01-b49f-463b-ba0a-8e90c3709e4e None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:40.871528 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 192/396] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:05:40.871528 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2257af01-b49f-463b-ba0a-8e90c3709e4e None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:40.928172 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94107bd2-b554-4839-9f58-bfbf8f7f1760 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:40.967820 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e05675-04d9-44e9-9d42-65e8a138e934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:40.969182 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e05675-04d9-44e9-9d42-65e8a138e934 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:40.969759 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e05675-04d9-44e9-9d42-65e8a138e934 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:40.970273 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e05675-04d9-44e9-9d42-65e8a138e934 None admin] PATH_INFO: `/v3/projects/21cd4117dcd346b8bcb3e353ff8d228a` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:40.971945 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e05675-04d9-44e9-9d42-65e8a138e934 None admin] RBAC: Authorizing `identity:get_project(project_id=21cd4117dcd346b8bcb3e353ff8d228a)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:40.974237 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e05675-04d9-44e9-9d42-65e8a138e934 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:40.976642 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 205/397] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jun 21 18:05:40 2022] GET /identity/v3/projects/21cd4117dcd346b8bcb3e353ff8d228a => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:40.977563 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2257af01-b49f-463b-ba0a-8e90c3709e4e None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:40.978452 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60830fef-741f-4cd8-bdbb-80dd0c1ef091 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:40.980223 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60830fef-741f-4cd8-bdbb-80dd0c1ef091 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:40.981211 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60830fef-741f-4cd8-bdbb-80dd0c1ef091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:41.002970 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-60830fef-741f-4cd8-bdbb-80dd0c1ef091 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:41.004532 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-60830fef-741f-4cd8-bdbb-80dd0c1ef091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:41.014515 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4599d696-825e-4fed-a7a2-c2b6e68ac466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:41.015067 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4599d696-825e-4fed-a7a2-c2b6e68ac466 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:41.015302 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4599d696-825e-4fed-a7a2-c2b6e68ac466 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:41.015527 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4599d696-825e-4fed-a7a2-c2b6e68ac466 None admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:41.016279 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4599d696-825e-4fed-a7a2-c2b6e68ac466 None admin] RBAC: Adding query filter params (name=member) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:41.016610 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4599d696-825e-4fed-a7a2-c2b6e68ac466 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:41.018350 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4599d696-825e-4fed-a7a2-c2b6e68ac466 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:41.027536 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 193/398] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jun 21 18:05:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:41.028509 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4599d696-825e-4fed-a7a2-c2b6e68ac466 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:41.060335 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:41.078963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:41.090014 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 206/399] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:05:41.091154 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60830fef-741f-4cd8-bdbb-80dd0c1ef091 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:41.134444 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4599d696-825e-4fed-a7a2-c2b6e68ac466 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:41.176145 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5298d76d-ce59-477e-8657-84150f876462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:41.177187 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5298d76d-ce59-477e-8657-84150f876462 None admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:41.177471 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5298d76d-ce59-477e-8657-84150f876462 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:41.177745 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5298d76d-ce59-477e-8657-84150f876462 None admin] PATH_INFO: `/v3/projects/21cd4117dcd346b8bcb3e353ff8d228a/groups/3d53537d1bfe4cf0bc60529db50c1c6a/roles/482dcc9ba0fd400cad2d193e3e895d4c` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:41.180190 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5298d76d-ce59-477e-8657-84150f876462 None admin] RBAC: Authorizing `identity:create_grant(project_id=21cd4117dcd346b8bcb3e353ff8d228a, group_id=3d53537d1bfe4cf0bc60529db50c1c6a, role_id=482dcc9ba0fd400cad2d193e3e895d4c)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:41.182701 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5298d76d-ce59-477e-8657-84150f876462 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:41.222158 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60830fef-741f-4cd8-bdbb-80dd0c1ef091 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:41.259725 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7289fb-1ef9-4f60-abb7-8cafe23b4653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:41.260640 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7289fb-1ef9-4f60-abb7-8cafe23b4653 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:41.260930 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7289fb-1ef9-4f60-abb7-8cafe23b4653 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:41.261207 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7289fb-1ef9-4f60-abb7-8cafe23b4653 None admin] PATH_INFO: `/v3/groups/3d53537d1bfe4cf0bc60529db50c1c6a` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:41.262455 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7289fb-1ef9-4f60-abb7-8cafe23b4653 None admin] RBAC: Authorizing `identity:get_group(group_id=3d53537d1bfe4cf0bc60529db50c1c6a)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:41.263892 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7289fb-1ef9-4f60-abb7-8cafe23b4653 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:41.270114 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 207/400] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Jun 21 18:05:41 2022] GET /identity/v3/groups/3d53537d1bfe4cf0bc60529db50c1c6a => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:41.270114 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7289fb-1ef9-4f60-abb7-8cafe23b4653 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:41.279328 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 194/401] 10.222.0.23 () {58 vars in 1442 bytes} [Tue Jun 21 18:05:41 2022] PUT /identity/v3/projects/21cd4117dcd346b8bcb3e353ff8d228a/groups/3d53537d1bfe4cf0bc60529db50c1c6a/roles/482dcc9ba0fd400cad2d193e3e895d4c => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:05:41.280120 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5298d76d-ce59-477e-8657-84150f876462 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:41.404068 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:41.415072 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5298d76d-ce59-477e-8657-84150f876462 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:41.455568 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:41.458577 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c6681b-6775-44f2-8df4-edee5c583033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:41.459659 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c6681b-6775-44f2-8df4-edee5c583033 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:41.460136 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c6681b-6775-44f2-8df4-edee5c583033 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:41.460601 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c6681b-6775-44f2-8df4-edee5c583033 None admin] PATH_INFO: `/v3/projects/53e988a4b11648fcb59e0748d5dbf37b` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:41.462724 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c6681b-6775-44f2-8df4-edee5c583033 None admin] RBAC: Authorizing `identity:get_project(project_id=53e988a4b11648fcb59e0748d5dbf37b)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:41.465174 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c6681b-6775-44f2-8df4-edee5c583033 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:41.467148 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 195/402] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jun 21 18:05:41 2022] GET /identity/v3/projects/53e988a4b11648fcb59e0748d5dbf37b => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:41.474002 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c6681b-6775-44f2-8df4-edee5c583033 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:41.503045 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7289fb-1ef9-4f60-abb7-8cafe23b4653 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:41.541704 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeae0f9d-6c20-4543-932f-676f2d5f835a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:41.542766 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae0f9d-6c20-4543-932f-676f2d5f835a None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:41.543260 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae0f9d-6c20-4543-932f-676f2d5f835a None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:41.543782 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae0f9d-6c20-4543-932f-676f2d5f835a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:41.554903 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeae0f9d-6c20-4543-932f-676f2d5f835a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:05:41.565991 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-aeae0f9d-6c20-4543-932f-676f2d5f835a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:05:41.565991 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:41.565991 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:41.565991 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:41.565991 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:41.565991 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:41.565991 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:41.565991 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:41.565991 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:41.565991 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:41.565991 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:41.565991 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:41.565991 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:05:41.565991 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:05:41.565991 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:05:41.565991 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:05:41.566963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:05:41.566963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:05:41.566963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:05:41.566963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:05:41.566963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:41.566963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:41.566963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:41.566963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:41.566963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:41.566963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:41.566963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:41.566963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:41.566963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:41.566963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:41.566963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:41.566963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:41.567807 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:41.567807 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:41.567807 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:41.567807 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:41.567807 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:05:41.567807 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:05:41.567807 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:05:41.567807 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:05:41.567807 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:05:41.567807 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:05:41.567807 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:05:41.567807 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:05:41.570683 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 208/403] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jun 21 18:05:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:41.580180 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeae0f9d-6c20-4543-932f-676f2d5f835a None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:41.593463 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c6681b-6775-44f2-8df4-edee5c583033 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:41.637530 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b8b722-8dd4-4dbd-afec-77c79cb82265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:41.639411 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b8b722-8dd4-4dbd-afec-77c79cb82265 None admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:41.639936 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b8b722-8dd4-4dbd-afec-77c79cb82265 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:41.640493 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b8b722-8dd4-4dbd-afec-77c79cb82265 None admin] PATH_INFO: `/v3/projects/53e988a4b11648fcb59e0748d5dbf37b/groups/3d53537d1bfe4cf0bc60529db50c1c6a/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:41.643626 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b8b722-8dd4-4dbd-afec-77c79cb82265 None admin] RBAC: Authorizing `identity:create_grant(project_id=53e988a4b11648fcb59e0748d5dbf37b, group_id=3d53537d1bfe4cf0bc60529db50c1c6a, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:41.645772 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b8b722-8dd4-4dbd-afec-77c79cb82265 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:41.705762 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeae0f9d-6c20-4543-932f-676f2d5f835a None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:41.733685 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:41.742482 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b091e42f-adb2-419c-b560-50bbc76ac9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:41.743199 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b091e42f-adb2-419c-b560-50bbc76ac9ef None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:41.743489 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b091e42f-adb2-419c-b560-50bbc76ac9ef None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:41.743769 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b091e42f-adb2-419c-b560-50bbc76ac9ef None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:41.745483 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b091e42f-adb2-419c-b560-50bbc76ac9ef None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:41.745859 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b091e42f-adb2-419c-b560-50bbc76ac9ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:41.747299 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b091e42f-adb2-419c-b560-50bbc76ac9ef None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:41.760611 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 209/404] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jun 21 18:05:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:41.762746 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b091e42f-adb2-419c-b560-50bbc76ac9ef None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:41.807479 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 196/405] 10.222.0.23 () {58 vars in 1442 bytes} [Tue Jun 21 18:05:41 2022] PUT /identity/v3/projects/53e988a4b11648fcb59e0748d5dbf37b/groups/3d53537d1bfe4cf0bc60529db50c1c6a/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:05:41.852771 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd84978-a585-437c-bd86-bf2cea62df7c None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:41.854683 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd84978-a585-437c-bd86-bf2cea62df7c None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:41.854980 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd84978-a585-437c-bd86-bf2cea62df7c None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:41.859135 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 197/406] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:41 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:05:41.870639 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b30ac-62fb-43a7-ab7e-be8e23852447 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:41.871219 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b30ac-62fb-43a7-ab7e-be8e23852447 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:41.871604 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b30ac-62fb-43a7-ab7e-be8e23852447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:41.891913 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-186b30ac-62fb-43a7-ab7e-be8e23852447 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:41.893326 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-186b30ac-62fb-43a7-ab7e-be8e23852447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:41.909750 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:41.924430 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b091e42f-adb2-419c-b560-50bbc76ac9ef None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:42.004763 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6114a1-8d34-48e0-b7c5-c5e54de50649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:42.005571 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6114a1-8d34-48e0-b7c5-c5e54de50649 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:42.008258 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6114a1-8d34-48e0-b7c5-c5e54de50649 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:42.008566 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6114a1-8d34-48e0-b7c5-c5e54de50649 None admin] PATH_INFO: `/v3/groups/3d53537d1bfe4cf0bc60529db50c1c6a` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:42.010249 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6114a1-8d34-48e0-b7c5-c5e54de50649 None admin] RBAC: Authorizing `identity:get_group(group_id=3d53537d1bfe4cf0bc60529db50c1c6a)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:42.012482 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6114a1-8d34-48e0-b7c5-c5e54de50649 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:42.015662 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 198/407] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:05:42.016464 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 210/408] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Jun 21 18:05:41 2022] GET /identity/v3/groups/3d53537d1bfe4cf0bc60529db50c1c6a => generated 226 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:05:42.022066 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b30ac-62fb-43a7-ab7e-be8e23852447 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:42.039410 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6114a1-8d34-48e0-b7c5-c5e54de50649 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:42.123130 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:42.168270 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b30ac-62fb-43a7-ab7e-be8e23852447 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:42.173074 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6114a1-8d34-48e0-b7c5-c5e54de50649 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:42.223618 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b2a1a0-4925-44eb-9aa8-a567291f4e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:42.224440 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b2a1a0-4925-44eb-9aa8-a567291f4e2f None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:42.224726 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b2a1a0-4925-44eb-9aa8-a567291f4e2f None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:42.225007 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b2a1a0-4925-44eb-9aa8-a567291f4e2f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:42.229989 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9307e2-a447-4b55-a0c3-f47f7f8a6c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:42.230857 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9307e2-a447-4b55-a0c3-f47f7f8a6c82 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:42.231144 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9307e2-a447-4b55-a0c3-f47f7f8a6c82 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:42.231413 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9307e2-a447-4b55-a0c3-f47f7f8a6c82 None admin] PATH_INFO: `/v3/projects/53e988a4b11648fcb59e0748d5dbf37b` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:42.235882 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9307e2-a447-4b55-a0c3-f47f7f8a6c82 None admin] RBAC: Authorizing `identity:get_project(project_id=53e988a4b11648fcb59e0748d5dbf37b)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:42.237377 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9307e2-a447-4b55-a0c3-f47f7f8a6c82 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:42.239115 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 199/409] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jun 21 18:05:42 2022] GET /identity/v3/projects/53e988a4b11648fcb59e0748d5dbf37b => generated 302 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:42.244792 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9307e2-a447-4b55-a0c3-f47f7f8a6c82 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:42.252659 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b2a1a0-4925-44eb-9aa8-a567291f4e2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:05:42.261711 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-b7b2a1a0-4925-44eb-9aa8-a567291f4e2f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:05:42.261711 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:42.261711 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:42.261711 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:42.261711 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:42.261711 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:42.261711 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:42.261711 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:42.261711 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:42.261711 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:42.261711 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:42.261711 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:42.261711 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:05:42.261711 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:05:42.261711 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:05:42.261711 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:05:42.262652 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:05:42.262652 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:05:42.262652 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:05:42.262652 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:05:42.262652 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:42.262652 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:42.262652 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:42.262652 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:42.262652 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:42.262652 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:42.262652 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:42.262652 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:42.262652 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:42.262652 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:42.262652 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:42.262652 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:42.263473 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:42.263473 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:42.263473 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:42.263473 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:42.263473 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:05:42.263473 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:05:42.263473 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:05:42.263473 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:05:42.263473 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:05:42.263473 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:05:42.263473 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:05:42.263473 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:05:42.264790 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:42.266576 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 211/410] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jun 21 18:05:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:42.268154 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cc1caf-ff86-4cbc-9109-d0e29749e37e None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:42.268659 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cc1caf-ff86-4cbc-9109-d0e29749e37e None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:42.269127 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cc1caf-ff86-4cbc-9109-d0e29749e37e None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:42.270686 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 212/411] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:05:42.275568 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cc1caf-ff86-4cbc-9109-d0e29749e37e None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:42.368774 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9307e2-a447-4b55-a0c3-f47f7f8a6c82 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:42.387932 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cc1caf-ff86-4cbc-9109-d0e29749e37e None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:42.405433 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:42.416451 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88410898-9703-4b95-a7b2-eaed54c3750d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:42.417977 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88410898-9703-4b95-a7b2-eaed54c3750d None admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:42.418269 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88410898-9703-4b95-a7b2-eaed54c3750d None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:42.418554 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88410898-9703-4b95-a7b2-eaed54c3750d None admin] PATH_INFO: `/v3/projects/53e988a4b11648fcb59e0748d5dbf37b/groups/3d53537d1bfe4cf0bc60529db50c1c6a/roles/482dcc9ba0fd400cad2d193e3e895d4c` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:42.421772 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88410898-9703-4b95-a7b2-eaed54c3750d None admin] RBAC: Authorizing `identity:create_grant(project_id=53e988a4b11648fcb59e0748d5dbf37b, group_id=3d53537d1bfe4cf0bc60529db50c1c6a, role_id=482dcc9ba0fd400cad2d193e3e895d4c)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:42.423577 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88410898-9703-4b95-a7b2-eaed54c3750d None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:42.425728 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ce54b7-e275-4a8d-a920-88e121e576f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:42.426644 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce54b7-e275-4a8d-a920-88e121e576f0 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:42.427169 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce54b7-e275-4a8d-a920-88e121e576f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:42.427634 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce54b7-e275-4a8d-a920-88e121e576f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:42.428654 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ce54b7-e275-4a8d-a920-88e121e576f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:42.429030 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ce54b7-e275-4a8d-a920-88e121e576f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:42.430456 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ce54b7-e275-4a8d-a920-88e121e576f0 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:42.445675 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 213/412] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:05:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:42.447243 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf54e93-7aad-4ed3-859c-b8f579d816c7 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:42.450242 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf54e93-7aad-4ed3-859c-b8f579d816c7 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:42.450242 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf54e93-7aad-4ed3-859c-b8f579d816c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:42.465970 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-eaf54e93-7aad-4ed3-859c-b8f579d816c7 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:42.467218 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-eaf54e93-7aad-4ed3-859c-b8f579d816c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:42.507313 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 200/413] 10.222.0.23 () {58 vars in 1442 bytes} [Tue Jun 21 18:05:42 2022] PUT /identity/v3/projects/53e988a4b11648fcb59e0748d5dbf37b/groups/3d53537d1bfe4cf0bc60529db50c1c6a/roles/482dcc9ba0fd400cad2d193e3e895d4c => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:05:42.507944 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88410898-9703-4b95-a7b2-eaed54c3750d None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:42.564915 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 214/414] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:05:42.627590 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:42.629921 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf54e93-7aad-4ed3-859c-b8f579d816c7 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:42.674861 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88410898-9703-4b95-a7b2-eaed54c3750d None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:42.722982 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2b496a-3dca-4b16-873a-9f4d420f220c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:42.723878 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2b496a-3dca-4b16-873a-9f4d420f220c None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:42.724172 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2b496a-3dca-4b16-873a-9f4d420f220c None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:42.724617 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2b496a-3dca-4b16-873a-9f4d420f220c None admin] PATH_INFO: `/v3/groups/efb3e72636cc4d62818b957d52674420` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:42.725835 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2b496a-3dca-4b16-873a-9f4d420f220c None admin] RBAC: Authorizing `identity:get_group(group_id=efb3e72636cc4d62818b957d52674420)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:42.727293 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2b496a-3dca-4b16-873a-9f4d420f220c None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:42.729415 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 201/415] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Jun 21 18:05:42 2022] GET /identity/v3/groups/efb3e72636cc4d62818b957d52674420 => generated 229 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:42.745113 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2b496a-3dca-4b16-873a-9f4d420f220c None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:42.844238 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf54e93-7aad-4ed3-859c-b8f579d816c7 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:42.862957 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:42.879487 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2b496a-3dca-4b16-873a-9f4d420f220c None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:42.920752 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3676282-2795-4c37-bdac-6f3e7201a2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:42.921976 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3676282-2795-4c37-bdac-6f3e7201a2e8 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:42.922407 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3676282-2795-4c37-bdac-6f3e7201a2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:42.922961 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3676282-2795-4c37-bdac-6f3e7201a2e8 None admin] PATH_INFO: `/v3/projects/cecad7a8a62644cabfc30fc28ce80d90` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:42.924402 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3676282-2795-4c37-bdac-6f3e7201a2e8 None admin] RBAC: Authorizing `identity:get_project(project_id=cecad7a8a62644cabfc30fc28ce80d90)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:42.926130 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3676282-2795-4c37-bdac-6f3e7201a2e8 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:42.926913 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df8bea4-5a8e-49a2-88d0-79f86f8342bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:42.927840 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df8bea4-5a8e-49a2-88d0-79f86f8342bd None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:42.928193 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df8bea4-5a8e-49a2-88d0-79f86f8342bd None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:42.928565 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df8bea4-5a8e-49a2-88d0-79f86f8342bd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:42.929330 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 202/416] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jun 21 18:05:42 2022] GET /identity/v3/projects/cecad7a8a62644cabfc30fc28ce80d90 => generated 344 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:42.936441 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3676282-2795-4c37-bdac-6f3e7201a2e8 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:42.956686 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df8bea4-5a8e-49a2-88d0-79f86f8342bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:05:42.965048 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-2df8bea4-5a8e-49a2-88d0-79f86f8342bd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:05:42.965048 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:42.965048 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:42.965048 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:42.965048 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:42.965048 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:42.965048 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:42.965048 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:42.965048 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:42.965048 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:42.965048 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:42.965048 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:42.965048 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:05:42.965048 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:05:42.965048 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:05:42.965048 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:05:42.965963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:05:42.965963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:05:42.965963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:05:42.965963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:05:42.965963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:42.965963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:42.965963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:42.965963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:42.965963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:42.965963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:42.965963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:42.965963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:42.965963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:42.965963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:42.965963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:42.965963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:42.966828 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:42.966828 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:42.966828 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:42.966828 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:42.966828 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:05:42.966828 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:05:42.966828 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:05:42.966828 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:05:42.966828 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:05:42.966828 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:05:42.966828 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:05:42.966828 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:05:42.966828 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 215/417] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:05:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:42.972926 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df8bea4-5a8e-49a2-88d0-79f86f8342bd None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:42.979068 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:43.137261 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3676282-2795-4c37-bdac-6f3e7201a2e8 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:43.155210 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df8bea4-5a8e-49a2-88d0-79f86f8342bd None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:43.178750 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8f5244-6eec-4ec9-8832-952c7847911e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:43.179921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8f5244-6eec-4ec9-8832-952c7847911e None admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:43.180279 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8f5244-6eec-4ec9-8832-952c7847911e None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:43.180656 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8f5244-6eec-4ec9-8832-952c7847911e None admin] PATH_INFO: `/v3/projects/cecad7a8a62644cabfc30fc28ce80d90/groups/efb3e72636cc4d62818b957d52674420/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:43.183625 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8f5244-6eec-4ec9-8832-952c7847911e None admin] RBAC: Authorizing `identity:create_grant(project_id=cecad7a8a62644cabfc30fc28ce80d90, group_id=efb3e72636cc4d62818b957d52674420, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:43.185129 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8f5244-6eec-4ec9-8832-952c7847911e None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:43.215014 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce538b45-fab0-44db-8ed1-28f156a2cb43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:43.215701 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce538b45-fab0-44db-8ed1-28f156a2cb43 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:43.215984 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce538b45-fab0-44db-8ed1-28f156a2cb43 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:43.216264 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce538b45-fab0-44db-8ed1-28f156a2cb43 None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:43.217052 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce538b45-fab0-44db-8ed1-28f156a2cb43 None admin] RBAC: Adding query filter params (name=member) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:43.217465 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce538b45-fab0-44db-8ed1-28f156a2cb43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:43.218971 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce538b45-fab0-44db-8ed1-28f156a2cb43 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:43.230064 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 216/418] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jun 21 18:05:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:05:43.235557 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce538b45-fab0-44db-8ed1-28f156a2cb43 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:43.267946 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 203/419] 10.222.0.23 () {58 vars in 1442 bytes} [Tue Jun 21 18:05:42 2022] PUT /identity/v3/projects/cecad7a8a62644cabfc30fc28ce80d90/groups/efb3e72636cc4d62818b957d52674420/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:05:43.283059 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:43.394582 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce538b45-fab0-44db-8ed1-28f156a2cb43 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:43.445230 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e270313-5ab8-4c24-b821-3f4a1dba6183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:43.445926 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e270313-5ab8-4c24-b821-3f4a1dba6183 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:43.446214 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e270313-5ab8-4c24-b821-3f4a1dba6183 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:43.446494 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e270313-5ab8-4c24-b821-3f4a1dba6183 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:43.529975 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e270313-5ab8-4c24-b821-3f4a1dba6183 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:43.531535 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e270313-5ab8-4c24-b821-3f4a1dba6183 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:43.561784 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:43.639434 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 217/420] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:05:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:05:43.644753 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8f5244-6eec-4ec9-8832-952c7847911e None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:43.733207 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:43.738364 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c362b2f-06bd-4838-9861-0e00e037a3d9 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:43.738662 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c362b2f-06bd-4838-9861-0e00e037a3d9 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:43.738946 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c362b2f-06bd-4838-9861-0e00e037a3d9 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:43.740591 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 218/421] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:05:43.746375 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c9b7d-ad00-4f59-8b96-132bc5465b89 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:43.746669 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c9b7d-ad00-4f59-8b96-132bc5465b89 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:43.746999 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c9b7d-ad00-4f59-8b96-132bc5465b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:43.766171 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8f5244-6eec-4ec9-8832-952c7847911e None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:43.793466 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-ce3c9b7d-ad00-4f59-8b96-132bc5465b89 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:43.794687 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-ce3c9b7d-ad00-4f59-8b96-132bc5465b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:43.801661 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3db242f-f3f2-4f30-bec3-c530611aceb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:43.802537 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db242f-f3f2-4f30-bec3-c530611aceb4 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:43.802824 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db242f-f3f2-4f30-bec3-c530611aceb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:43.803105 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db242f-f3f2-4f30-bec3-c530611aceb4 None admin] PATH_INFO: `/v3/projects/21cd4117dcd346b8bcb3e353ff8d228a` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:43.804542 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3db242f-f3f2-4f30-bec3-c530611aceb4 None admin] RBAC: Authorizing `identity:get_project(project_id=21cd4117dcd346b8bcb3e353ff8d228a)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:43.806083 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3db242f-f3f2-4f30-bec3-c530611aceb4 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:43.808145 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 204/422] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jun 21 18:05:43 2022] GET /identity/v3/projects/21cd4117dcd346b8bcb3e353ff8d228a => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:43.813260 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3db242f-f3f2-4f30-bec3-c530611aceb4 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:43.921265 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3db242f-f3f2-4f30-bec3-c530611aceb4 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:43.922674 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 219/423] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:05:43.947429 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c9b7d-ad00-4f59-8b96-132bc5465b89 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:43.958937 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b014ead-e68e-48e2-8006-51789c9c06c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:43.960220 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b014ead-e68e-48e2-8006-51789c9c06c9 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:43.960685 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b014ead-e68e-48e2-8006-51789c9c06c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:43.961092 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b014ead-e68e-48e2-8006-51789c9c06c9 None admin] PATH_INFO: `/v3/groups/3d53537d1bfe4cf0bc60529db50c1c6a` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:43.962598 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b014ead-e68e-48e2-8006-51789c9c06c9 None admin] RBAC: Authorizing `identity:get_group(group_id=3d53537d1bfe4cf0bc60529db50c1c6a)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:43.968419 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b014ead-e68e-48e2-8006-51789c9c06c9 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:43.968621 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 205/424] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Jun 21 18:05:43 2022] GET /identity/v3/groups/3d53537d1bfe4cf0bc60529db50c1c6a => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:05:43.975558 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b014ead-e68e-48e2-8006-51789c9c06c9 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:43.992811 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:44.075951 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:44.110026 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b014ead-e68e-48e2-8006-51789c9c06c9 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:44.147070 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b6442d-d7ed-4e72-8122-1d06fb88d645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:44.148590 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b6442d-d7ed-4e72-8122-1d06fb88d645 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:44.149110 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b6442d-d7ed-4e72-8122-1d06fb88d645 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:44.149586 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b6442d-d7ed-4e72-8122-1d06fb88d645 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:44.151561 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b6442d-d7ed-4e72-8122-1d06fb88d645 None admin] RBAC: Adding query filter params (group.id=3d53537d1bfe4cf0bc60529db50c1c6a, role.id=c467f10d3dba4f2dbfd42068f42f74b6, scope.project.id=21cd4117dcd346b8bcb3e353ff8d228a) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:44.152084 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b6442d-d7ed-4e72-8122-1d06fb88d645 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:44.153884 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b6442d-d7ed-4e72-8122-1d06fb88d645 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:44.164739 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 206/425] 10.222.0.23 () {56 vars in 1376 bytes} [Tue Jun 21 18:05:43 2022] GET /identity/v3/role_assignments?group.id=3d53537d1bfe4cf0bc60529db50c1c6a&scope.project.id=21cd4117dcd346b8bcb3e353ff8d228a&role.id=c467f10d3dba4f2dbfd42068f42f74b6 => generated 611 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:05:44.191982 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c9b7d-ad00-4f59-8b96-132bc5465b89 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:44.254591 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba40b743-9dbf-49ba-99f6-82ee432b046e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:44.255526 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba40b743-9dbf-49ba-99f6-82ee432b046e None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:44.255813 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba40b743-9dbf-49ba-99f6-82ee432b046e None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:44.256092 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba40b743-9dbf-49ba-99f6-82ee432b046e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:44.275990 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba40b743-9dbf-49ba-99f6-82ee432b046e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:05:44.284787 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-ba40b743-9dbf-49ba-99f6-82ee432b046e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:05:44.284787 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:44.284787 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:44.284787 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:44.284787 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:44.284787 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:44.284787 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:44.284787 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:44.284787 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:44.284787 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:44.284787 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:44.284787 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:44.284787 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:05:44.284787 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:05:44.284787 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:05:44.284787 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:05:44.285763 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:05:44.285763 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:05:44.285763 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:05:44.285763 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:05:44.285763 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:44.285763 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:44.285763 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:44.285763 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:44.285763 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:44.285763 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:44.285763 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:44.285763 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:44.285763 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:44.285763 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:44.285763 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:44.285763 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:44.286577 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:44.286577 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:44.286577 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:44.286577 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:44.286577 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:05:44.286577 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:05:44.286577 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:05:44.286577 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:05:44.286577 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:05:44.286577 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:05:44.286577 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:05:44.286577 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:05:44.286577 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 220/426] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jun 21 18:05:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:44.292576 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b6442d-d7ed-4e72-8122-1d06fb88d645 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:44.403879 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:44.437955 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b6442d-d7ed-4e72-8122-1d06fb88d645 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:44.459256 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaac4f24-e3d2-4b13-9285-45dac54f0c90 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:44.459564 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaac4f24-e3d2-4b13-9285-45dac54f0c90 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:44.459750 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaac4f24-e3d2-4b13-9285-45dac54f0c90 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:44.461527 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 221/427] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:05:44.467097 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612cb269-d635-4d5c-ab38-8d6938eeae3e None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:44.467390 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612cb269-d635-4d5c-ab38-8d6938eeae3e None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:44.467671 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612cb269-d635-4d5c-ab38-8d6938eeae3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:44.475175 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6d426e-3c12-460d-884f-ee5283b3d56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:44.476166 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d426e-3c12-460d-884f-ee5283b3d56e None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:44.476693 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d426e-3c12-460d-884f-ee5283b3d56e None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:44.477200 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d426e-3c12-460d-884f-ee5283b3d56e None admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:44.478694 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6d426e-3c12-460d-884f-ee5283b3d56e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:44.479204 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6d426e-3c12-460d-884f-ee5283b3d56e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:44.480911 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6d426e-3c12-460d-884f-ee5283b3d56e None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:44.489471 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-612cb269-d635-4d5c-ab38-8d6938eeae3e None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:44.490660 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 207/428] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jun 21 18:05:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:44.492793 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-612cb269-d635-4d5c-ab38-8d6938eeae3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:44.497533 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6d426e-3c12-460d-884f-ee5283b3d56e None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:44.594124 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 222/429] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:05:44.627889 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6d426e-3c12-460d-884f-ee5283b3d56e None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:44.628919 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612cb269-d635-4d5c-ab38-8d6938eeae3e None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:44.671398 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d851164-e408-4dad-805a-eb4b46b2719e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:44.672151 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d851164-e408-4dad-805a-eb4b46b2719e None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:44.672472 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d851164-e408-4dad-805a-eb4b46b2719e None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:44.672807 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d851164-e408-4dad-805a-eb4b46b2719e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:44.676931 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:44.707836 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:44.762431 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d851164-e408-4dad-805a-eb4b46b2719e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:44.763865 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d851164-e408-4dad-805a-eb4b46b2719e None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:44.792692 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612cb269-d635-4d5c-ab38-8d6938eeae3e None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:44.835050 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb070967-6580-47a5-b3fd-61a552a7d510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:44.836209 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb070967-6580-47a5-b3fd-61a552a7d510 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:44.836719 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb070967-6580-47a5-b3fd-61a552a7d510 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:44.838284 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb070967-6580-47a5-b3fd-61a552a7d510 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:44.854488 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb070967-6580-47a5-b3fd-61a552a7d510 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:05:44.863890 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-eb070967-6580-47a5-b3fd-61a552a7d510 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:05:44.863890 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:44.863890 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:44.863890 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:44.863890 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:44.863890 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:44.863890 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:44.863890 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:44.863890 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:44.863890 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:44.863890 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:44.863890 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:44.863890 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:05:44.863890 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:05:44.863890 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:05:44.863890 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:05:44.864854 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:05:44.864854 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:05:44.864854 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:05:44.864854 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:05:44.864854 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:44.864854 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:44.864854 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:44.864854 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:44.864854 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:44.864854 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:44.864854 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:44.864854 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:44.864854 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:44.864854 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:44.864854 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:44.864854 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:44.865693 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:44.865693 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:44.865693 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:44.865693 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:44.865693 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:05:44.865693 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:05:44.865693 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:05:44.865693 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:05:44.865693 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:05:44.865693 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:05:44.865693 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:05:44.865693 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:05:44.865693 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 223/430] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:05:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:44.872381 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb070967-6580-47a5-b3fd-61a552a7d510 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:44.913677 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 208/431] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:05:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 21 18:05:44.919758 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d851164-e408-4dad-805a-eb4b46b2719e None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:44.964591 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:45.001443 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb070967-6580-47a5-b3fd-61a552a7d510 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:45.017020 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:45.048179 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d851164-e408-4dad-805a-eb4b46b2719e None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:45.064954 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56c9443-61b7-47e5-8b28-864f6e1a8095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:45.066149 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56c9443-61b7-47e5-8b28-864f6e1a8095 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:45.066580 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56c9443-61b7-47e5-8b28-864f6e1a8095 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:45.067005 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56c9443-61b7-47e5-8b28-864f6e1a8095 None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:45.068093 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56c9443-61b7-47e5-8b28-864f6e1a8095 None admin] RBAC: Adding query filter params (name=member) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:45.068573 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56c9443-61b7-47e5-8b28-864f6e1a8095 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:45.070447 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56c9443-61b7-47e5-8b28-864f6e1a8095 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:45.083730 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 224/432] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jun 21 18:05:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:45.089968 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56c9443-61b7-47e5-8b28-864f6e1a8095 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:45.094180 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c15b1c-d275-4f7d-8de5-878fe5526210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:45.095916 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c15b1c-d275-4f7d-8de5-878fe5526210 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:45.096202 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c15b1c-d275-4f7d-8de5-878fe5526210 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:45.096512 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c15b1c-d275-4f7d-8de5-878fe5526210 None admin] PATH_INFO: `/v3/projects/21cd4117dcd346b8bcb3e353ff8d228a` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:45.098025 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c15b1c-d275-4f7d-8de5-878fe5526210 None admin] RBAC: Authorizing `identity:get_project(project_id=21cd4117dcd346b8bcb3e353ff8d228a)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:45.099475 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c15b1c-d275-4f7d-8de5-878fe5526210 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:45.101480 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 209/433] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jun 21 18:05:44 2022] GET /identity/v3/projects/21cd4117dcd346b8bcb3e353ff8d228a => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:05:45.109071 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c15b1c-d275-4f7d-8de5-878fe5526210 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:45.208846 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56c9443-61b7-47e5-8b28-864f6e1a8095 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:45.237343 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c15b1c-d275-4f7d-8de5-878fe5526210 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:45.238992 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:45.249454 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e238978-1d61-488f-a931-5fe63bd91cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:45.250114 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e238978-1d61-488f-a931-5fe63bd91cb6 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:45.250401 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e238978-1d61-488f-a931-5fe63bd91cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:45.250677 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e238978-1d61-488f-a931-5fe63bd91cb6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:45.274667 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db19d7f-ec7c-4cce-8be4-717191767e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:45.276750 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db19d7f-ec7c-4cce-8be4-717191767e37 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:45.277041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db19d7f-ec7c-4cce-8be4-717191767e37 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:45.277316 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db19d7f-ec7c-4cce-8be4-717191767e37 None admin] PATH_INFO: `/v3/groups/3d53537d1bfe4cf0bc60529db50c1c6a` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:45.278523 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db19d7f-ec7c-4cce-8be4-717191767e37 None admin] RBAC: Authorizing `identity:get_group(group_id=3d53537d1bfe4cf0bc60529db50c1c6a)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:45.279951 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db19d7f-ec7c-4cce-8be4-717191767e37 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:45.281623 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 210/434] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Jun 21 18:05:45 2022] GET /identity/v3/groups/3d53537d1bfe4cf0bc60529db50c1c6a => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:45.282783 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef47ad-8fdc-421b-9b2c-7429b2383bea None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:45.283106 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef47ad-8fdc-421b-9b2c-7429b2383bea None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:45.283385 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef47ad-8fdc-421b-9b2c-7429b2383bea None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:45.288497 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 211/435] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:05:45.289284 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aef47ad-8fdc-421b-9b2c-7429b2383bea None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:45.302418 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:45.330913 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e238978-1d61-488f-a931-5fe63bd91cb6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:45.332373 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e238978-1d61-488f-a931-5fe63bd91cb6 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:45.402867 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aef47ad-8fdc-421b-9b2c-7429b2383bea None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:45.439596 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a439905b-618e-4696-bad1-2f70ab4aff1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:45.440329 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a439905b-618e-4696-bad1-2f70ab4aff1b None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:45.440639 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a439905b-618e-4696-bad1-2f70ab4aff1b None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:45.441043 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a439905b-618e-4696-bad1-2f70ab4aff1b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:45.441744 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a439905b-618e-4696-bad1-2f70ab4aff1b None admin] RBAC: Adding query filter params (group.id=3d53537d1bfe4cf0bc60529db50c1c6a, role.id=482dcc9ba0fd400cad2d193e3e895d4c, scope.project.id=21cd4117dcd346b8bcb3e353ff8d228a) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:45.442056 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a439905b-618e-4696-bad1-2f70ab4aff1b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:45.443608 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a439905b-618e-4696-bad1-2f70ab4aff1b None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:45.444656 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 225/436] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:05:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 21 18:05:45.445809 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc1dec1-4868-4bb0-b36d-5b3307796db1 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:45.446098 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc1dec1-4868-4bb0-b36d-5b3307796db1 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:45.446464 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc1dec1-4868-4bb0-b36d-5b3307796db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:45.456854 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 212/437] 10.222.0.23 () {56 vars in 1376 bytes} [Tue Jun 21 18:05:45 2022] GET /identity/v3/role_assignments?group.id=3d53537d1bfe4cf0bc60529db50c1c6a&scope.project.id=21cd4117dcd346b8bcb3e353ff8d228a&role.id=482dcc9ba0fd400cad2d193e3e895d4c => generated 611 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:45.458564 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a439905b-618e-4696-bad1-2f70ab4aff1b None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:45.468915 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-bbc1dec1-4868-4bb0-b36d-5b3307796db1 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:45.469597 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-bbc1dec1-4868-4bb0-b36d-5b3307796db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:45.513752 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:45.560497 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 226/438] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:05:45.584317 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc1dec1-4868-4bb0-b36d-5b3307796db1 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:45.587282 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:45.616339 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a439905b-618e-4696-bad1-2f70ab4aff1b None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:45.656199 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c220c7ab-4a34-4c9c-91b8-4df59f50dc51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:45.657453 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c220c7ab-4a34-4c9c-91b8-4df59f50dc51 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:45.657739 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c220c7ab-4a34-4c9c-91b8-4df59f50dc51 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:45.658017 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c220c7ab-4a34-4c9c-91b8-4df59f50dc51 None admin] PATH_INFO: `/v3/projects/53e988a4b11648fcb59e0748d5dbf37b` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:45.659226 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c220c7ab-4a34-4c9c-91b8-4df59f50dc51 None admin] RBAC: Authorizing `identity:get_project(project_id=53e988a4b11648fcb59e0748d5dbf37b)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:45.660695 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c220c7ab-4a34-4c9c-91b8-4df59f50dc51 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:45.662713 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 213/439] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jun 21 18:05:45 2022] GET /identity/v3/projects/53e988a4b11648fcb59e0748d5dbf37b => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:45.668840 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c220c7ab-4a34-4c9c-91b8-4df59f50dc51 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:45.726929 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc1dec1-4868-4bb0-b36d-5b3307796db1 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:45.768583 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6186b581-5069-4654-9d0c-2ec198a949f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:45.769409 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6186b581-5069-4654-9d0c-2ec198a949f5 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:45.769686 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6186b581-5069-4654-9d0c-2ec198a949f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:45.770456 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6186b581-5069-4654-9d0c-2ec198a949f5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:45.784168 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6186b581-5069-4654-9d0c-2ec198a949f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:05:45.793059 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-6186b581-5069-4654-9d0c-2ec198a949f5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:05:45.793059 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:45.793059 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:45.793059 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:45.793059 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:45.793059 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:45.793059 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:45.793059 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:45.793059 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:45.793059 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:45.793059 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:45.793059 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:45.793059 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:05:45.793059 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:05:45.793059 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:05:45.793059 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:05:45.793982 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:05:45.793982 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:05:45.793982 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:05:45.793982 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:05:45.793982 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:45.793982 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:45.793982 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:45.793982 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:45.793982 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:45.793982 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:45.793982 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:45.793982 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:45.793982 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:45.793982 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:45.793982 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:45.793982 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:45.794807 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:45.794807 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:45.794807 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:45.794807 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:45.794807 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:05:45.794807 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:05:45.794807 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:05:45.794807 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:05:45.794807 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:05:45.794807 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:05:45.794807 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:05:45.794807 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:05:45.794807 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 227/440] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jun 21 18:05:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:05:45.795485 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa08f781-b4fc-4fe4-a8df-213546c47bd0 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:45.795485 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa08f781-b4fc-4fe4-a8df-213546c47bd0 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:45.795756 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa08f781-b4fc-4fe4-a8df-213546c47bd0 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:45.797264 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 228/441] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:05:45.799604 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa08f781-b4fc-4fe4-a8df-213546c47bd0 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:45.805708 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c220c7ab-4a34-4c9c-91b8-4df59f50dc51 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:45.808617 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:45.856158 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee88f030-7a73-4222-9f6a-d30d644b2896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:45.857384 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee88f030-7a73-4222-9f6a-d30d644b2896 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:45.857816 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee88f030-7a73-4222-9f6a-d30d644b2896 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:45.858238 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee88f030-7a73-4222-9f6a-d30d644b2896 None admin] PATH_INFO: `/v3/groups/3d53537d1bfe4cf0bc60529db50c1c6a` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:45.859666 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee88f030-7a73-4222-9f6a-d30d644b2896 None admin] RBAC: Authorizing `identity:get_group(group_id=3d53537d1bfe4cf0bc60529db50c1c6a)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:45.861155 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee88f030-7a73-4222-9f6a-d30d644b2896 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:45.863110 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 214/442] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Jun 21 18:05:45 2022] GET /identity/v3/groups/3d53537d1bfe4cf0bc60529db50c1c6a => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:45.864306 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2d0cb5-ab3e-4a9e-9655-428e1392d74e None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:45.864617 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2d0cb5-ab3e-4a9e-9655-428e1392d74e None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:45.864903 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2d0cb5-ab3e-4a9e-9655-428e1392d74e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:45.886093 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-fe2d0cb5-ab3e-4a9e-9655-428e1392d74e None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:45.886641 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:45.887208 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-fe2d0cb5-ab3e-4a9e-9655-428e1392d74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:45.929112 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa08f781-b4fc-4fe4-a8df-213546c47bd0 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:45.957111 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 215/443] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:05:45.957956 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2d0cb5-ab3e-4a9e-9655-428e1392d74e None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:45.975755 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62217f2-5f2a-4f63-b3be-b9e1a2ab5424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:45.976462 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62217f2-5f2a-4f63-b3be-b9e1a2ab5424 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:45.976753 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62217f2-5f2a-4f63-b3be-b9e1a2ab5424 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:45.977034 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62217f2-5f2a-4f63-b3be-b9e1a2ab5424 None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:45.977791 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62217f2-5f2a-4f63-b3be-b9e1a2ab5424 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:45.978130 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62217f2-5f2a-4f63-b3be-b9e1a2ab5424 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:45.979809 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62217f2-5f2a-4f63-b3be-b9e1a2ab5424 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:45.992270 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 229/444] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jun 21 18:05:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:05:45.992270 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62217f2-5f2a-4f63-b3be-b9e1a2ab5424 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:46.059959 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:46.063904 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2d0cb5-ab3e-4a9e-9655-428e1392d74e None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:46.092873 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62217f2-5f2a-4f63-b3be-b9e1a2ab5424 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:46.101652 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64ec875-fcbf-4713-8398-7a1620b7b845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:46.102312 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ec875-fcbf-4713-8398-7a1620b7b845 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:46.102594 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ec875-fcbf-4713-8398-7a1620b7b845 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:46.102873 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ec875-fcbf-4713-8398-7a1620b7b845 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:46.104351 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64ec875-fcbf-4713-8398-7a1620b7b845 None admin] RBAC: Adding query filter params (group.id=3d53537d1bfe4cf0bc60529db50c1c6a, role.id=c467f10d3dba4f2dbfd42068f42f74b6, scope.project.id=53e988a4b11648fcb59e0748d5dbf37b) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:46.104692 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64ec875-fcbf-4713-8398-7a1620b7b845 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:46.106114 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64ec875-fcbf-4713-8398-7a1620b7b845 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:46.111945 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 216/445] 10.222.0.23 () {56 vars in 1376 bytes} [Tue Jun 21 18:05:45 2022] GET /identity/v3/role_assignments?group.id=3d53537d1bfe4cf0bc60529db50c1c6a&scope.project.id=53e988a4b11648fcb59e0748d5dbf37b&role.id=c467f10d3dba4f2dbfd42068f42f74b6 => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:46.112689 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64ec875-fcbf-4713-8398-7a1620b7b845 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:46.116437 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:46.128134 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145e1adb-08fe-4e4f-a5bd-6e66a4a1bfec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:46.129102 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e1adb-08fe-4e4f-a5bd-6e66a4a1bfec None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:46.129390 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e1adb-08fe-4e4f-a5bd-6e66a4a1bfec None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:46.129694 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e1adb-08fe-4e4f-a5bd-6e66a4a1bfec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:46.145098 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145e1adb-08fe-4e4f-a5bd-6e66a4a1bfec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:05:46.155060 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-145e1adb-08fe-4e4f-a5bd-6e66a4a1bfec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:05:46.155060 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:46.155060 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:46.155060 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:46.155060 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:46.155060 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:46.155060 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:46.155060 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:46.155060 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:46.155060 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:46.155060 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:46.155060 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:46.155060 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:05:46.155060 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:05:46.155060 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:05:46.155060 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:05:46.156165 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:05:46.156165 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:05:46.156165 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:05:46.156165 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:05:46.156165 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:46.156165 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:46.156165 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:46.156165 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:46.156165 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:46.156165 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:46.156165 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:46.156165 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:46.156165 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:46.156165 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:46.156165 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:46.156165 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:46.157026 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:46.157026 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:46.157026 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:46.157026 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:46.157026 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:05:46.157026 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:05:46.157026 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:05:46.157026 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:05:46.157026 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:05:46.157026 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:05:46.157026 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:05:46.157026 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:05:46.157026 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 230/446] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jun 21 18:05:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:46.162339 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145e1adb-08fe-4e4f-a5bd-6e66a4a1bfec None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:46.250244 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64ec875-fcbf-4713-8398-7a1620b7b845 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:46.280289 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145e1adb-08fe-4e4f-a5bd-6e66a4a1bfec None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:46.294940 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c51152d-a5e0-44e1-be34-983bbf25ac19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:46.294940 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51152d-a5e0-44e1-be34-983bbf25ac19 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:46.294940 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51152d-a5e0-44e1-be34-983bbf25ac19 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:46.294940 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51152d-a5e0-44e1-be34-983bbf25ac19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:46.325863 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc04b0e-b730-4fe5-a37d-cfab1e8fca92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:46.326547 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc04b0e-b730-4fe5-a37d-cfab1e8fca92 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:46.327969 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc04b0e-b730-4fe5-a37d-cfab1e8fca92 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:46.327969 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc04b0e-b730-4fe5-a37d-cfab1e8fca92 None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:46.327969 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc04b0e-b730-4fe5-a37d-cfab1e8fca92 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:46.328211 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc04b0e-b730-4fe5-a37d-cfab1e8fca92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:46.329541 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc04b0e-b730-4fe5-a37d-cfab1e8fca92 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:46.336281 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:46.339911 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 231/447] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:05:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:05:46.345724 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc04b0e-b730-4fe5-a37d-cfab1e8fca92 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:46.381744 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c51152d-a5e0-44e1-be34-983bbf25ac19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:46.383130 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c51152d-a5e0-44e1-be34-983bbf25ac19 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:46.399328 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:46.479000 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc04b0e-b730-4fe5-a37d-cfab1e8fca92 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:46.500126 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 217/448] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:05:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:05:46.505794 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c51152d-a5e0-44e1-be34-983bbf25ac19 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:46.514937 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea2b20f-9c1c-4598-872f-2bafb23e2508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:46.515587 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea2b20f-9c1c-4598-872f-2bafb23e2508 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:46.515874 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea2b20f-9c1c-4598-872f-2bafb23e2508 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:46.516152 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea2b20f-9c1c-4598-872f-2bafb23e2508 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:46.567097 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:46.590179 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea2b20f-9c1c-4598-872f-2bafb23e2508 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:46.591836 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea2b20f-9c1c-4598-872f-2bafb23e2508 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:46.607529 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c51152d-a5e0-44e1-be34-983bbf25ac19 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:46.637465 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:46.641315 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89538661-54fa-4312-85af-f137f9a4a6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:46.642188 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89538661-54fa-4312-85af-f137f9a4a6be None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:46.642471 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89538661-54fa-4312-85af-f137f9a4a6be None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:46.642748 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89538661-54fa-4312-85af-f137f9a4a6be None admin] PATH_INFO: `/v3/projects/53e988a4b11648fcb59e0748d5dbf37b` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:46.643957 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89538661-54fa-4312-85af-f137f9a4a6be None admin] RBAC: Authorizing `identity:get_project(project_id=53e988a4b11648fcb59e0748d5dbf37b)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:46.645417 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89538661-54fa-4312-85af-f137f9a4a6be None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:46.647126 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 218/449] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jun 21 18:05:46 2022] GET /identity/v3/projects/53e988a4b11648fcb59e0748d5dbf37b => generated 302 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:05:46.653319 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89538661-54fa-4312-85af-f137f9a4a6be None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:46.695139 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 232/450] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:05:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 21 18:05:46.696227 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea2b20f-9c1c-4598-872f-2bafb23e2508 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:46.757203 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89538661-54fa-4312-85af-f137f9a4a6be None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:46.786852 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:46.792797 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ab20fa-5ee2-4c21-81fe-154e32cb9be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:46.793679 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ab20fa-5ee2-4c21-81fe-154e32cb9be2 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:46.793959 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ab20fa-5ee2-4c21-81fe-154e32cb9be2 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:46.794238 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ab20fa-5ee2-4c21-81fe-154e32cb9be2 None admin] PATH_INFO: `/v3/groups/3d53537d1bfe4cf0bc60529db50c1c6a` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:46.795514 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ab20fa-5ee2-4c21-81fe-154e32cb9be2 None admin] RBAC: Authorizing `identity:get_group(group_id=3d53537d1bfe4cf0bc60529db50c1c6a)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:46.797105 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ab20fa-5ee2-4c21-81fe-154e32cb9be2 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:46.798745 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 219/451] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Jun 21 18:05:46 2022] GET /identity/v3/groups/3d53537d1bfe4cf0bc60529db50c1c6a => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:05:46.803565 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea2b20f-9c1c-4598-872f-2bafb23e2508 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:46.804614 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ab20fa-5ee2-4c21-81fe-154e32cb9be2 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:46.839425 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9b57d0-0c63-40f4-aa52-30fe15247c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:46.840356 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b57d0-0c63-40f4-aa52-30fe15247c59 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:46.840672 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b57d0-0c63-40f4-aa52-30fe15247c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:46.840996 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b57d0-0c63-40f4-aa52-30fe15247c59 None admin] PATH_INFO: `/v3/projects/cecad7a8a62644cabfc30fc28ce80d90` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:46.842308 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9b57d0-0c63-40f4-aa52-30fe15247c59 None admin] RBAC: Authorizing `identity:get_project(project_id=cecad7a8a62644cabfc30fc28ce80d90)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:46.843778 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9b57d0-0c63-40f4-aa52-30fe15247c59 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:46.845582 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 233/452] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jun 21 18:05:46 2022] GET /identity/v3/projects/cecad7a8a62644cabfc30fc28ce80d90 => generated 344 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:46.851291 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9b57d0-0c63-40f4-aa52-30fe15247c59 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:46.867908 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:46.916713 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ab20fa-5ee2-4c21-81fe-154e32cb9be2 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:46.950375 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ffb82e-60db-47a6-b8bc-9e1c2a828703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:46.950573 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9b57d0-0c63-40f4-aa52-30fe15247c59 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:46.951030 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ffb82e-60db-47a6-b8bc-9e1c2a828703 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:46.951317 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ffb82e-60db-47a6-b8bc-9e1c2a828703 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:46.951637 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ffb82e-60db-47a6-b8bc-9e1c2a828703 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:46.952483 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ffb82e-60db-47a6-b8bc-9e1c2a828703 None admin] RBAC: Adding query filter params (group.id=3d53537d1bfe4cf0bc60529db50c1c6a, role.id=482dcc9ba0fd400cad2d193e3e895d4c, scope.project.id=53e988a4b11648fcb59e0748d5dbf37b) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:46.952796 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ffb82e-60db-47a6-b8bc-9e1c2a828703 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:46.954215 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ffb82e-60db-47a6-b8bc-9e1c2a828703 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:46.960098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 220/453] 10.222.0.23 () {56 vars in 1376 bytes} [Tue Jun 21 18:05:46 2022] GET /identity/v3/role_assignments?group.id=3d53537d1bfe4cf0bc60529db50c1c6a&scope.project.id=53e988a4b11648fcb59e0748d5dbf37b&role.id=482dcc9ba0fd400cad2d193e3e895d4c => generated 611 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:05:46.986073 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d384bb77-09e1-40e6-9b4c-fe1aa2465880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:46.986952 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d384bb77-09e1-40e6-9b4c-fe1aa2465880 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:46.987270 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d384bb77-09e1-40e6-9b4c-fe1aa2465880 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:46.987520 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d384bb77-09e1-40e6-9b4c-fe1aa2465880 None admin] PATH_INFO: `/v3/groups/efb3e72636cc4d62818b957d52674420` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:46.988826 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d384bb77-09e1-40e6-9b4c-fe1aa2465880 None admin] RBAC: Authorizing `identity:get_group(group_id=efb3e72636cc4d62818b957d52674420)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:46.990295 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d384bb77-09e1-40e6-9b4c-fe1aa2465880 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:46.991987 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 234/454] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Jun 21 18:05:46 2022] GET /identity/v3/groups/efb3e72636cc4d62818b957d52674420 => generated 229 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:05:46.998737 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ffb82e-60db-47a6-b8bc-9e1c2a828703 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:47.066473 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:47.106902 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ffb82e-60db-47a6-b8bc-9e1c2a828703 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:47.145246 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b551b754-20ce-4fbb-93c2-508121692fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:47.146028 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b551b754-20ce-4fbb-93c2-508121692fbf None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:47.146313 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b551b754-20ce-4fbb-93c2-508121692fbf None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:47.146589 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b551b754-20ce-4fbb-93c2-508121692fbf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:47.147445 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b551b754-20ce-4fbb-93c2-508121692fbf None admin] RBAC: Adding query filter params (group.id=efb3e72636cc4d62818b957d52674420, role.id=745ed9065a8143e384ebbd694b29635c, scope.project.id=cecad7a8a62644cabfc30fc28ce80d90) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:47.147755 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b551b754-20ce-4fbb-93c2-508121692fbf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:47.149252 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b551b754-20ce-4fbb-93c2-508121692fbf None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:47.156692 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 221/455] 10.222.0.23 () {56 vars in 1376 bytes} [Tue Jun 21 18:05:46 2022] GET /identity/v3/role_assignments?group.id=efb3e72636cc4d62818b957d52674420&scope.project.id=cecad7a8a62644cabfc30fc28ce80d90&role.id=745ed9065a8143e384ebbd694b29635c => generated 611 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:49.783053 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6648845c-4a3a-4664-a2d6-375c511f2440 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:49.784112 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6648845c-4a3a-4664-a2d6-375c511f2440 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:49.784720 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6648845c-4a3a-4664-a2d6-375c511f2440 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:49.786710 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 235/456] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:05:49.792768 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaccf9c-5dcb-4acd-b806-67fa2bd6a6ed None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:49.793298 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaccf9c-5dcb-4acd-b806-67fa2bd6a6ed None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:49.793298 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaccf9c-5dcb-4acd-b806-67fa2bd6a6ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:49.822068 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-cdaccf9c-5dcb-4acd-b806-67fa2bd6a6ed None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:49.823467 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-cdaccf9c-5dcb-4acd-b806-67fa2bd6a6ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:49.941319 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 222/457] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:05:49.982055 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6648845c-4a3a-4664-a2d6-375c511f2440 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:50.009355 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f1f492-86ee-418b-a1cc-9239d9515eeb None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:50.010507 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f1f492-86ee-418b-a1cc-9239d9515eeb None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:50.010985 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f1f492-86ee-418b-a1cc-9239d9515eeb None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:50.012882 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 223/458] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:05:50.018498 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2081c54-826e-419d-89fa-eaf59dbf8a33 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:50.018793 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2081c54-826e-419d-89fa-eaf59dbf8a33 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:50.019078 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2081c54-826e-419d-89fa-eaf59dbf8a33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:50.053288 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-a2081c54-826e-419d-89fa-eaf59dbf8a33 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:50.054405 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-a2081c54-826e-419d-89fa-eaf59dbf8a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:50.156202 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:50.159502 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:50.174762 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 224/459] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:05:50.202196 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2081c54-826e-419d-89fa-eaf59dbf8a33 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:50.216123 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6648845c-4a3a-4664-a2d6-375c511f2440 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:50.261064 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395f04bc-1e0d-4438-b295-a1ed2ef26a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:50.261877 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f04bc-1e0d-4438-b295-a1ed2ef26a8a None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:50.262155 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f04bc-1e0d-4438-b295-a1ed2ef26a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:50.262428 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f04bc-1e0d-4438-b295-a1ed2ef26a8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:50.273687 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395f04bc-1e0d-4438-b295-a1ed2ef26a8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:50.275348 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395f04bc-1e0d-4438-b295-a1ed2ef26a8a None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:50.296455 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-395f04bc-1e0d-4438-b295-a1ed2ef26a8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:05:50.296455 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:50.296455 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:50.296455 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:50.296455 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:50.296455 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:50.296455 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:50.296455 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:50.296455 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:50.296455 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:50.296455 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:50.296455 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:50.296455 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:05:50.296455 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:05:50.296455 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:05:50.297380 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:05:50.297380 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:50.297380 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:50.297380 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:50.297380 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:50.297380 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:50.297380 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:50.297380 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:50.297380 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:50.297380 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:50.297380 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:50.297380 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:50.297380 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:50.297380 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:50.297380 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:50.297380 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:50.297380 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:50.298259 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:05:50.298259 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:05:50.298259 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:05:50.298259 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:05:50.298259 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 236/460] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:05:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:50.302958 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12bee02-3435-4966-8c75-aeb452c91a3c None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:50.302958 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12bee02-3435-4966-8c75-aeb452c91a3c None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:50.302958 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12bee02-3435-4966-8c75-aeb452c91a3c None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:50.302958 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 237/461] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:05:50.306750 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12bee02-3435-4966-8c75-aeb452c91a3c None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:50.341432 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2081c54-826e-419d-89fa-eaf59dbf8a33 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:50.383555 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8907fc-f15f-4554-9a25-ec378954e8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:50.384652 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8907fc-f15f-4554-9a25-ec378954e8c9 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:50.384943 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8907fc-f15f-4554-9a25-ec378954e8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:50.385222 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8907fc-f15f-4554-9a25-ec378954e8c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:50.398717 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8907fc-f15f-4554-9a25-ec378954e8c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:50.401344 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8907fc-f15f-4554-9a25-ec378954e8c9 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:50.426641 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-bc8907fc-f15f-4554-9a25-ec378954e8c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:05:50.426641 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:50.426641 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:50.426641 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:50.426641 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:50.426641 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:50.426641 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:50.426641 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:50.426641 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:50.426641 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:50.426641 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:50.426641 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:50.426641 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:05:50.426641 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:05:50.426641 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:05:50.427580 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:05:50.427580 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:50.427580 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:50.427580 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:50.427580 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:50.427580 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:50.427580 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:50.427580 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:50.427580 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:50.427580 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:50.427580 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:50.427580 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:50.427580 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:50.427580 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:50.427580 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:50.427580 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:50.427580 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:50.428520 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:05:50.428520 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:05:50.428520 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:05:50.428520 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:05:50.428735 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12bee02-3435-4966-8c75-aeb452c91a3c None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:50.428822 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 225/462] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:05:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:05:50.429569 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decc2554-1d0f-4751-bcfc-760a16e1a4ce None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:50.429861 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decc2554-1d0f-4751-bcfc-760a16e1a4ce None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:50.430140 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decc2554-1d0f-4751-bcfc-760a16e1a4ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:50.453092 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-decc2554-1d0f-4751-bcfc-760a16e1a4ce None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:50.454333 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-decc2554-1d0f-4751-bcfc-760a16e1a4ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:50.468116 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef69734-7e59-4c24-bb3f-24d50bb0d5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:50.468821 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef69734-7e59-4c24-bb3f-24d50bb0d5c4 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:50.469105 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef69734-7e59-4c24-bb3f-24d50bb0d5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:50.469531 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef69734-7e59-4c24-bb3f-24d50bb0d5c4 None admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:50.470114 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef69734-7e59-4c24-bb3f-24d50bb0d5c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:50.470419 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef69734-7e59-4c24-bb3f-24d50bb0d5c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:50.471807 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef69734-7e59-4c24-bb3f-24d50bb0d5c4 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:50.474037 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:50.485733 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 238/463] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:05:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:50.487259 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4443f09-f6b3-413a-b608-b039dadb4742 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:50.487753 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4443f09-f6b3-413a-b608-b039dadb4742 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:50.488208 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4443f09-f6b3-413a-b608-b039dadb4742 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:50.489888 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 239/464] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:05:50.491005 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4443f09-f6b3-413a-b608-b039dadb4742 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:50.503684 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:50.544438 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 226/465] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:05:50.544438 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decc2554-1d0f-4751-bcfc-760a16e1a4ce None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:50.613187 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4443f09-f6b3-413a-b608-b039dadb4742 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:50.656237 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6445fe4-ce0f-4d3a-b11a-6a279e072c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:50.657100 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6445fe4-ce0f-4d3a-b11a-6a279e072c67 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:50.657385 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6445fe4-ce0f-4d3a-b11a-6a279e072c67 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:50.657661 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6445fe4-ce0f-4d3a-b11a-6a279e072c67 None admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:50.658389 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6445fe4-ce0f-4d3a-b11a-6a279e072c67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:50.658694 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6445fe4-ce0f-4d3a-b11a-6a279e072c67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:50.659731 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decc2554-1d0f-4751-bcfc-760a16e1a4ce None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:50.660072 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6445fe4-ce0f-4d3a-b11a-6a279e072c67 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:50.678673 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 240/466] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:05:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:50.679954 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ee00e-99a9-4144-888d-7dda9459ffd8 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:50.680246 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ee00e-99a9-4144-888d-7dda9459ffd8 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:50.680557 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ee00e-99a9-4144-888d-7dda9459ffd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:50.697791 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-6e4ee00e-99a9-4144-888d-7dda9459ffd8 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:50.698996 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-6e4ee00e-99a9-4144-888d-7dda9459ffd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:50.699464 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debd6d6b-4ae7-48ea-88c1-4ab99bc15a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:50.700150 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd6d6b-4ae7-48ea-88c1-4ab99bc15a75 None admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:50.700460 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd6d6b-4ae7-48ea-88c1-4ab99bc15a75 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:50.700742 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd6d6b-4ae7-48ea-88c1-4ab99bc15a75 None admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:50.701441 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debd6d6b-4ae7-48ea-88c1-4ab99bc15a75 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:50.702872 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debd6d6b-4ae7-48ea-88c1-4ab99bc15a75 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:50.750152 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:50.792421 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 241/467] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:05:50.792421 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4ee00e-99a9-4144-888d-7dda9459ffd8 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:50.808485 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 227/468] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Jun 21 18:05:50 2022] POST /identity/v3/users => generated 246 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:05:50.809402 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debd6d6b-4ae7-48ea-88c1-4ab99bc15a75 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:50.851569 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:50.942350 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4ee00e-99a9-4144-888d-7dda9459ffd8 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:50.963739 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debd6d6b-4ae7-48ea-88c1-4ab99bc15a75 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:50.990242 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55a0c54-1d2c-4038-9093-b3ca04d11f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:50.991044 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a0c54-1d2c-4038-9093-b3ca04d11f26 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:50.991324 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a0c54-1d2c-4038-9093-b3ca04d11f26 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:50.991656 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a0c54-1d2c-4038-9093-b3ca04d11f26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:51.005747 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55a0c54-1d2c-4038-9093-b3ca04d11f26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:51.007051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55a0c54-1d2c-4038-9093-b3ca04d11f26 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:51.017155 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac8b84b-261a-4023-ace9-0a02a1e025d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:51.017847 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac8b84b-261a-4023-ace9-0a02a1e025d6 None admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:51.018130 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac8b84b-261a-4023-ace9-0a02a1e025d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:51.018404 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac8b84b-261a-4023-ace9-0a02a1e025d6 None admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:51.019110 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac8b84b-261a-4023-ace9-0a02a1e025d6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:51.020576 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac8b84b-261a-4023-ace9-0a02a1e025d6 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:51.026990 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-b55a0c54-1d2c-4038-9093-b3ca04d11f26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:05:51.026990 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:51.026990 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:51.026990 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:51.026990 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:51.026990 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:51.026990 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:51.026990 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:51.026990 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:51.026990 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:51.026990 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:51.026990 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:51.026990 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:05:51.026990 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:05:51.026990 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:05:51.028015 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:05:51.028015 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:51.028015 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:51.028015 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:51.028015 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:51.028015 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:51.028015 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:51.028015 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:51.028015 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:51.028015 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:51.028015 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:51.028015 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:51.028015 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:51.028015 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:51.028015 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:51.028015 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:51.028015 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:51.029175 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:05:51.029175 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:05:51.029175 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:05:51.029175 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:05:51.029175 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 242/469] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:05:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:51.029175 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55a0c54-1d2c-4038-9093-b3ca04d11f26 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:51.076819 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:51.102486 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 228/470] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Jun 21 18:05:50 2022] POST /identity/v3/users => generated 248 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:05:51.104448 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a65e1-2e39-445f-87e6-b36a8611ca54 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:51.104783 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a65e1-2e39-445f-87e6-b36a8611ca54 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:51.105082 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a65e1-2e39-445f-87e6-b36a8611ca54 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:51.106647 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 229/471] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:05:51.107265 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21a65e1-2e39-445f-87e6-b36a8611ca54 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:51.179261 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55a0c54-1d2c-4038-9093-b3ca04d11f26 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:51.229219 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af621b14-8dd9-48b1-b848-f9ad60c456b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:51.230787 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af621b14-8dd9-48b1-b848-f9ad60c456b5 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:51.231307 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af621b14-8dd9-48b1-b848-f9ad60c456b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:51.231802 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af621b14-8dd9-48b1-b848-f9ad60c456b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:51.231918 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:51.247938 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af621b14-8dd9-48b1-b848-f9ad60c456b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:51.250033 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af621b14-8dd9-48b1-b848-f9ad60c456b5 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:51.256957 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21a65e1-2e39-445f-87e6-b36a8611ca54 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:51.271635 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-af621b14-8dd9-48b1-b848-f9ad60c456b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:05:51.271635 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:51.271635 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:51.271635 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:51.271635 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:51.271635 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:51.271635 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:51.271635 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:51.271635 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:51.271635 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:51.271635 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:51.271635 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:51.271635 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:05:51.271635 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:05:51.271635 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:05:51.272607 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:05:51.272607 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:51.272607 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:51.272607 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:51.272607 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:51.272607 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:51.272607 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:51.272607 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:51.272607 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:51.272607 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:51.272607 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:51.272607 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:51.272607 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:51.272607 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:51.272607 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:51.272607 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:51.272607 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:51.273476 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:05:51.273476 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:05:51.273476 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:05:51.273476 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:05:51.273476 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 243/472] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:05:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:51.274198 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18585f0-76a6-4ea2-90ba-38bd3b48a667 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:51.274377 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18585f0-76a6-4ea2-90ba-38bd3b48a667 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:51.274632 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18585f0-76a6-4ea2-90ba-38bd3b48a667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:51.294119 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-b18585f0-76a6-4ea2-90ba-38bd3b48a667 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:51.295568 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-b18585f0-76a6-4ea2-90ba-38bd3b48a667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:51.316561 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe196e27-73bb-4f7a-a612-08c8c0ce3d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:51.316561 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe196e27-73bb-4f7a-a612-08c8c0ce3d69 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:51.316561 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe196e27-73bb-4f7a-a612-08c8c0ce3d69 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:51.316561 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe196e27-73bb-4f7a-a612-08c8c0ce3d69 None admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:51.317402 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe196e27-73bb-4f7a-a612-08c8c0ce3d69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:51.317823 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe196e27-73bb-4f7a-a612-08c8c0ce3d69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:51.319509 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe196e27-73bb-4f7a-a612-08c8c0ce3d69 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:51.333606 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 230/473] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:05:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:51.334508 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe196e27-73bb-4f7a-a612-08c8c0ce3d69 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:51.383474 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 244/474] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:05:51.384968 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18585f0-76a6-4ea2-90ba-38bd3b48a667 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:51.408908 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:51.452767 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe196e27-73bb-4f7a-a612-08c8c0ce3d69 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:51.489919 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d121b16-fb9e-432e-b0ea-2e8fc3ff18c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:51.490595 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d121b16-fb9e-432e-b0ea-2e8fc3ff18c4 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:51.490878 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d121b16-fb9e-432e-b0ea-2e8fc3ff18c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:51.491153 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d121b16-fb9e-432e-b0ea-2e8fc3ff18c4 None admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:51.491871 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d121b16-fb9e-432e-b0ea-2e8fc3ff18c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:51.492176 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d121b16-fb9e-432e-b0ea-2e8fc3ff18c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:51.494420 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d121b16-fb9e-432e-b0ea-2e8fc3ff18c4 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:51.503539 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18585f0-76a6-4ea2-90ba-38bd3b48a667 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:51.511534 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 231/475] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:05:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:51.511534 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d121b16-fb9e-432e-b0ea-2e8fc3ff18c4 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:51.511534 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:51.548136 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5648fe1-0b07-40ad-8a46-9d9685d7ca6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:51.549197 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5648fe1-0b07-40ad-8a46-9d9685d7ca6c None admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:51.549750 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5648fe1-0b07-40ad-8a46-9d9685d7ca6c None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:51.550293 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5648fe1-0b07-40ad-8a46-9d9685d7ca6c None admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:51.551245 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5648fe1-0b07-40ad-8a46-9d9685d7ca6c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:51.553002 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5648fe1-0b07-40ad-8a46-9d9685d7ca6c None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:51.616300 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d121b16-fb9e-432e-b0ea-2e8fc3ff18c4 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:51.652047 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a77689-0969-45cf-a48a-bf184eab4dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:51.652671 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a77689-0969-45cf-a48a-bf184eab4dfa None admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:51.652959 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a77689-0969-45cf-a48a-bf184eab4dfa None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:51.653234 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a77689-0969-45cf-a48a-bf184eab4dfa None admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:51.653985 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a77689-0969-45cf-a48a-bf184eab4dfa None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:51.655353 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a77689-0969-45cf-a48a-bf184eab4dfa None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:51.713051 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.sql.core [None req-61a77689-0969-45cf-a48a-bf184eab4dfa None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 21 18:05:51.713051 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 21 18:05:51.713051 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [parameters: {'id': '5798655b14354a58bda9135f7ef8c9fb', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 21 18:05:51.713051 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80615) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 21 18:05:51.730066 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-61a77689-0969-45cf-a48a-bf184eab4dfa None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 21 18:05:51.730066 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:51.730066 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 21 18:05:51.730066 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 21 18:05:51.730066 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 21 18:05:51.730066 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 21 18:05:51.730066 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 21 18:05:51.730066 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application result = self._query(query) Jun 21 18:05:51.730066 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 21 18:05:51.730066 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application conn.query(q) Jun 21 18:05:51.730066 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 21 18:05:51.730066 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 21 18:05:51.730066 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 21 18:05:51.730066 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application result.read() Jun 21 18:05:51.730066 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 21 18:05:51.730986 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 21 18:05:51.730986 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 21 18:05:51.730986 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 21 18:05:51.730986 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 21 18:05:51.730986 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 21 18:05:51.730986 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 21 18:05:51.730986 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 21 18:05:51.730986 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 21 18:05:51.730986 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:05:51.730986 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 21 18:05:51.730986 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:05:51.730986 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:51.730986 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 21 18:05:51.730986 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 21 18:05:51.730986 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 21 18:05:51.730986 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return ref.to_dict() Jun 21 18:05:51.730986 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 21 18:05:51.730986 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application next(self.gen) Jun 21 18:05:51.732628 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 21 18:05:51.732628 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application yield resource Jun 21 18:05:51.732628 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 21 18:05:51.732628 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application next(self.gen) Jun 21 18:05:51.732628 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 21 18:05:51.732628 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application self.session.rollback() Jun 21 18:05:51.732628 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 21 18:05:51.732628 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application self.force_reraise() Jun 21 18:05:51.732628 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 21 18:05:51.732628 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise self.value Jun 21 18:05:51.732628 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 21 18:05:51.732628 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 21 18:05:51.732628 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 21 18:05:51.732628 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application session.commit() Jun 21 18:05:51.732628 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 21 18:05:51.732628 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 21 18:05:51.732628 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 21 18:05:51.733516 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application self._prepare_impl() Jun 21 18:05:51.733516 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 21 18:05:51.733516 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application self.session.flush() Jun 21 18:05:51.733516 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 21 18:05:51.733516 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application self._flush(objects) Jun 21 18:05:51.733516 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 21 18:05:51.733516 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 21 18:05:51.733516 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 21 18:05:51.733516 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application compat.raise_( Jun 21 18:05:51.733516 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 21 18:05:51.733516 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception Jun 21 18:05:51.733516 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 21 18:05:51.733516 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application flush_context.execute() Jun 21 18:05:51.733516 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 21 18:05:51.733516 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rec.execute(self) Jun 21 18:05:51.733516 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 21 18:05:51.733516 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 21 18:05:51.734413 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 21 18:05:51.734413 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 21 18:05:51.734413 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 21 18:05:51.734413 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 21 18:05:51.734413 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 21 18:05:51.734413 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 21 18:05:51.734413 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 21 18:05:51.734413 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 21 18:05:51.734413 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 21 18:05:51.734413 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 21 18:05:51.734413 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 21 18:05:51.734413 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 21 18:05:51.734413 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 21 18:05:51.734413 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 21 18:05:51.734413 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 21 18:05:51.734413 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception Jun 21 18:05:51.735240 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 21 18:05:51.735240 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 21 18:05:51.735240 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 21 18:05:51.735240 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 21 18:05:51.735240 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 21 18:05:51.735240 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application result = self._query(query) Jun 21 18:05:51.735240 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 21 18:05:51.735240 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application conn.query(q) Jun 21 18:05:51.735240 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 21 18:05:51.735240 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 21 18:05:51.735240 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 21 18:05:51.735240 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application result.read() Jun 21 18:05:51.735240 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 21 18:05:51.735240 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 21 18:05:51.735240 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 21 18:05:51.735240 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 21 18:05:51.737032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 21 18:05:51.737032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 21 18:05:51.737032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 21 18:05:51.737032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 21 18:05:51.737032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 21 18:05:51.737032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 21 18:05:51.737032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [parameters: {'id': '5798655b14354a58bda9135f7ef8c9fb', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 21 18:05:51.737032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 21 18:05:51.737032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:05:51.737032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:05:51.737032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:05:51.737032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:51.737032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:51.737032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:51.737032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:51.737032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:51.739369 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:51.739369 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:51.739369 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:51.739369 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:51.739369 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:51.739369 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:51.739369 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 21 18:05:51.739369 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 21 18:05:51.739369 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:51.739369 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:51.739369 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 21 18:05:51.739369 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 21 18:05:51.739369 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 21 18:05:51.739369 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 21 18:05:51.739369 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 21 18:05:51.739369 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:05:51.741525 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 232/476] 10.222.0.23 () {60 vars in 1131 bytes} [Tue Jun 21 18:05:51 2022] POST /identity/v3/roles => generated 144 bytes in 223 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:51.741525 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a77689-0969-45cf-a48a-bf184eab4dfa None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:51.764533 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 245/477] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Jun 21 18:05:51 2022] POST /identity/v3/users => generated 248 bytes in 380 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:05:51.765397 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5648fe1-0b07-40ad-8a46-9d9685d7ca6c None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:51.818910 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:51.881859 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a77689-0969-45cf-a48a-bf184eab4dfa None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:51.889888 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:51.956776 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5648fe1-0b07-40ad-8a46-9d9685d7ca6c None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:51.962056 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b12ae6-9a79-4f93-b096-da98329bb151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:51.962886 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b12ae6-9a79-4f93-b096-da98329bb151 None admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:51.963278 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b12ae6-9a79-4f93-b096-da98329bb151 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:51.963570 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b12ae6-9a79-4f93-b096-da98329bb151 None admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:51.964315 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b12ae6-9a79-4f93-b096-da98329bb151 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:51.965901 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b12ae6-9a79-4f93-b096-da98329bb151 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:52.007552 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6e9c60-3011-4eff-ad2e-2c10ac2b97c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:52.008494 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e9c60-3011-4eff-ad2e-2c10ac2b97c4 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:52.008780 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e9c60-3011-4eff-ad2e-2c10ac2b97c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:52.009062 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e9c60-3011-4eff-ad2e-2c10ac2b97c4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:52.021586 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6e9c60-3011-4eff-ad2e-2c10ac2b97c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:05:52.030413 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-3d6e9c60-3011-4eff-ad2e-2c10ac2b97c4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:05:52.030413 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:52.030413 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:52.030413 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:52.030413 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:52.030413 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:52.030413 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:52.030413 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:52.030413 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:52.030413 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:52.030413 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:52.030413 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:52.030413 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:05:52.030413 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:05:52.030413 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:05:52.030413 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:05:52.031436 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:05:52.031436 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:05:52.031436 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:05:52.031436 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:05:52.031436 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:52.031436 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:52.031436 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:52.031436 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:52.031436 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:52.031436 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:52.031436 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:52.031436 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:52.031436 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:52.031436 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:52.031436 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:52.031436 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:52.032272 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:52.032272 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:52.032272 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:52.032272 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:52.032272 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:05:52.032272 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:05:52.032272 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:05:52.032272 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:05:52.032272 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:05:52.032272 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:05:52.032272 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:05:52.032272 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:05:52.032272 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 246/478] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jun 21 18:05:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:52.041456 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6e9c60-3011-4eff-ad2e-2c10ac2b97c4 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:52.065697 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 233/479] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Jun 21 18:05:51 2022] POST /identity/v3/users => generated 249 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:05:52.178467 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6e9c60-3011-4eff-ad2e-2c10ac2b97c4 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:52.223918 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aba782b-0210-4126-9f27-e51fa9e6ee99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:52.225042 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba782b-0210-4126-9f27-e51fa9e6ee99 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:52.225538 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba782b-0210-4126-9f27-e51fa9e6ee99 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:52.226022 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba782b-0210-4126-9f27-e51fa9e6ee99 None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:52.227029 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aba782b-0210-4126-9f27-e51fa9e6ee99 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:52.227557 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aba782b-0210-4126-9f27-e51fa9e6ee99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:52.229796 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aba782b-0210-4126-9f27-e51fa9e6ee99 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:52.231948 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:52.243163 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 247/480] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jun 21 18:05:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:53.211433 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e4569c-bbbd-4db7-89c4-e52f9fd1fc48 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:53.211952 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e4569c-bbbd-4db7-89c4-e52f9fd1fc48 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:53.212052 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e4569c-bbbd-4db7-89c4-e52f9fd1fc48 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:53.214192 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 234/481] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:05:53.219035 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6981816-b308-4c45-a5d8-7c97749d3296 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:53.219478 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6981816-b308-4c45-a5d8-7c97749d3296 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:53.219647 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6981816-b308-4c45-a5d8-7c97749d3296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:53.239419 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-f6981816-b308-4c45-a5d8-7c97749d3296 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:53.240628 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-f6981816-b308-4c45-a5d8-7c97749d3296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:53.326511 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 248/482] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:05:53.350253 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e4569c-bbbd-4db7-89c4-e52f9fd1fc48 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:53.474413 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e4569c-bbbd-4db7-89c4-e52f9fd1fc48 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:53.498725 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:53.510764 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31403bd0-836a-439f-8d02-29c62849c24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:53.511579 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31403bd0-836a-439f-8d02-29c62849c24b None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:53.518953 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31403bd0-836a-439f-8d02-29c62849c24b None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:53.518953 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31403bd0-836a-439f-8d02-29c62849c24b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:53.522878 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31403bd0-836a-439f-8d02-29c62849c24b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:05:53.531277 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-31403bd0-836a-439f-8d02-29c62849c24b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:05:53.531277 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:53.531277 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:53.531277 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:53.531277 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:53.531277 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:53.531277 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:53.531277 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:53.531277 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:53.531277 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:53.531277 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:53.531277 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:53.531277 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:05:53.531277 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:05:53.531277 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:05:53.531277 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:05:53.532224 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:05:53.532224 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:05:53.532224 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:05:53.532224 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:05:53.532224 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:53.532224 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:53.532224 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:53.532224 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:53.532224 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:53.532224 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:53.532224 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:53.532224 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:53.532224 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:53.532224 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:53.532224 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:53.532224 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:53.533173 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:53.533173 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:53.533173 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:53.533173 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:53.533173 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:05:53.533173 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:05:53.533173 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:05:53.533173 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:05:53.533173 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:05:53.533173 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:05:53.533173 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:05:53.533173 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:05:53.533173 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 235/483] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jun 21 18:05:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:53.541573 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6981816-b308-4c45-a5d8-7c97749d3296 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:53.686845 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6981816-b308-4c45-a5d8-7c97749d3296 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:53.726339 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:53.736746 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6c466a-3833-4f93-8814-ad8a271fa2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:53.742197 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6c466a-3833-4f93-8814-ad8a271fa2c0 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:53.742197 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6c466a-3833-4f93-8814-ad8a271fa2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:53.742197 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6c466a-3833-4f93-8814-ad8a271fa2c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:53.742197 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6c466a-3833-4f93-8814-ad8a271fa2c0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:53.742197 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6c466a-3833-4f93-8814-ad8a271fa2c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:53.742197 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6c466a-3833-4f93-8814-ad8a271fa2c0 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:53.756031 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 249/484] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jun 21 18:05:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:53.761777 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31403bd0-836a-439f-8d02-29c62849c24b None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:53.796710 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd189625-d0b0-45d0-a68a-5b0e1665d06f None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:53.797014 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd189625-d0b0-45d0-a68a-5b0e1665d06f None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:53.797377 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd189625-d0b0-45d0-a68a-5b0e1665d06f None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:53.799064 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 250/485] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:05:53.804107 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7c673d-00e3-4454-84b4-8a2baa05644c None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:53.804415 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7c673d-00e3-4454-84b4-8a2baa05644c None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:53.804711 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7c673d-00e3-4454-84b4-8a2baa05644c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:53.823845 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-3e7c673d-00e3-4454-84b4-8a2baa05644c None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:53.825921 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-3e7c673d-00e3-4454-84b4-8a2baa05644c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:53.963567 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 251/486] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:05:53.966064 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31403bd0-836a-439f-8d02-29c62849c24b None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:54.004634 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8666e480-7e9e-4784-8b35-957aa20bd0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:54.005359 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666e480-7e9e-4784-8b35-957aa20bd0e6 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:54.005643 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666e480-7e9e-4784-8b35-957aa20bd0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:54.005919 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666e480-7e9e-4784-8b35-957aa20bd0e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:54.014004 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7c673d-00e3-4454-84b4-8a2baa05644c None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:54.033292 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8666e480-7e9e-4784-8b35-957aa20bd0e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:54.036207 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8666e480-7e9e-4784-8b35-957aa20bd0e6 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:54.068486 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-8666e480-7e9e-4784-8b35-957aa20bd0e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:05:54.068486 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:54.068486 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:54.068486 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:54.068486 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:54.068486 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:54.068486 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:54.068486 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:54.068486 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:54.068486 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:54.068486 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:54.068486 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:54.068486 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:05:54.068486 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:05:54.068486 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:05:54.069431 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:05:54.069431 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:54.069431 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:54.069431 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:54.069431 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:54.069431 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:54.069431 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:54.069431 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:54.069431 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:54.069431 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:54.069431 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:54.069431 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:54.069431 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:54.069431 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:54.069431 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:54.069431 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:54.069431 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:54.070321 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:05:54.070321 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:05:54.070321 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:05:54.070321 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:05:54.070960 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 236/487] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:05:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:54.076458 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8666e480-7e9e-4784-8b35-957aa20bd0e6 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:54.146826 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:54.200855 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7c673d-00e3-4454-84b4-8a2baa05644c None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:54.241588 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:54.248310 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8666e480-7e9e-4784-8b35-957aa20bd0e6 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:54.275853 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba2806e-f552-4dc9-9619-715e244ecd2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:54.276671 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2806e-f552-4dc9-9619-715e244ecd2a None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:54.276958 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2806e-f552-4dc9-9619-715e244ecd2a None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:54.277236 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2806e-f552-4dc9-9619-715e244ecd2a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:54.287600 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c03334-fd79-43b2-a5df-e7eb8d70f078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:54.288589 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c03334-fd79-43b2-a5df-e7eb8d70f078 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:54.289118 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c03334-fd79-43b2-a5df-e7eb8d70f078 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:54.289595 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c03334-fd79-43b2-a5df-e7eb8d70f078 None admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:54.291052 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c03334-fd79-43b2-a5df-e7eb8d70f078 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:54.291617 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c03334-fd79-43b2-a5df-e7eb8d70f078 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:54.293549 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c03334-fd79-43b2-a5df-e7eb8d70f078 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:54.301294 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba2806e-f552-4dc9-9619-715e244ecd2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:05:54.308549 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 237/488] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:05:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:54.316555 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-fba2806e-f552-4dc9-9619-715e244ecd2a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:05:54.316555 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:54.316555 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:54.316555 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:54.316555 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:54.316555 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:54.316555 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:54.316555 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:54.316555 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:54.316555 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:54.316555 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:54.316555 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:54.316555 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:05:54.316555 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:05:54.316555 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:05:54.316555 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:05:54.317503 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:05:54.317503 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:05:54.317503 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:05:54.317503 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:05:54.317503 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:54.317503 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:54.317503 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:54.317503 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:54.317503 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:54.317503 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:54.317503 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:54.317503 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:54.317503 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:54.317503 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:54.317503 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:54.317503 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:54.318337 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:54.318337 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:54.318337 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:54.318337 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:54.318337 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:05:54.318337 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:05:54.318337 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:05:54.318337 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:05:54.318337 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:05:54.318337 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:05:54.318337 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:05:54.318337 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:05:54.318337 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 252/489] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jun 21 18:05:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:05:54.319018 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c03334-fd79-43b2-a5df-e7eb8d70f078 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:54.321491 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba2806e-f552-4dc9-9619-715e244ecd2a None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:54.473977 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:54.483142 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c03334-fd79-43b2-a5df-e7eb8d70f078 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:54.485096 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba2806e-f552-4dc9-9619-715e244ecd2a None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:54.528862 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2f55eb-e327-4525-bce4-ae601519b427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:54.529696 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2f55eb-e327-4525-bce4-ae601519b427 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:54.530019 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2f55eb-e327-4525-bce4-ae601519b427 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:54.530366 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2f55eb-e327-4525-bce4-ae601519b427 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:54.531758 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abf0573-8351-4c14-8368-301c53089a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:54.532504 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abf0573-8351-4c14-8368-301c53089a70 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:54.533096 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abf0573-8351-4c14-8368-301c53089a70 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:54.533380 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abf0573-8351-4c14-8368-301c53089a70 None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:54.534795 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abf0573-8351-4c14-8368-301c53089a70 None admin] RBAC: Adding query filter params (name=service) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:54.535110 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abf0573-8351-4c14-8368-301c53089a70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:54.536519 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abf0573-8351-4c14-8368-301c53089a70 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:54.539728 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2f55eb-e327-4525-bce4-ae601519b427 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:54.541365 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2f55eb-e327-4525-bce4-ae601519b427 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:54.547095 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 253/490] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jun 21 18:05:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:54.548562 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6db2bc-07ba-4baa-aa7b-8565853a8110 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:54.548854 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6db2bc-07ba-4baa-aa7b-8565853a8110 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:54.549135 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6db2bc-07ba-4baa-aa7b-8565853a8110 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:54.550546 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 254/491] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:05:54.555468 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6db2bc-07ba-4baa-aa7b-8565853a8110 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:54.566765 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-cc2f55eb-e327-4525-bce4-ae601519b427 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 21 18:05:54.566765 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:54.566765 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:54.566765 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:54.566765 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:54.566765 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:54.566765 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:54.566765 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:54.566765 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:54.566765 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:54.566765 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:54.566765 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:54.566765 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:05:54.566765 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:05:54.566765 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:05:54.566765 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:05:54.567975 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:54.567975 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:54.567975 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:05:54.567975 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:05:54.567975 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:05:54.567975 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:05:54.567975 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:54.567975 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:54.567975 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:54.567975 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:54.567975 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:54.567975 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:54.567975 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:54.567975 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:54.567975 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:54.567975 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:54.567975 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:54.567975 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:54.570019 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:54.570019 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:54.570019 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:05:54.570019 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:05:54.570019 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:05:54.570019 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:05:54.570019 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:05:54.570019 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:05:54.570019 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 21 18:05:54.570019 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:05:54.576640 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 238/492] 10.222.0.23 () {56 vars in 1093 bytes} [Tue Jun 21 18:05:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:54.576640 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98552409-171c-49a9-9c0d-89e1cdac9a75 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:54.576640 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98552409-171c-49a9-9c0d-89e1cdac9a75 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:54.576640 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98552409-171c-49a9-9c0d-89e1cdac9a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:54.594042 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-98552409-171c-49a9-9c0d-89e1cdac9a75 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:54.595259 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-98552409-171c-49a9-9c0d-89e1cdac9a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:54.612055 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:54.675565 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 239/493] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:05:54.676434 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98552409-171c-49a9-9c0d-89e1cdac9a75 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:54.691043 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6db2bc-07ba-4baa-aa7b-8565853a8110 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:54.728217 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9b6358-f485-41d6-b5ae-66d21ceaaa61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:54.729128 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b6358-f485-41d6-b5ae-66d21ceaaa61 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:54.729414 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b6358-f485-41d6-b5ae-66d21ceaaa61 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:54.729692 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b6358-f485-41d6-b5ae-66d21ceaaa61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:54.741276 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9b6358-f485-41d6-b5ae-66d21ceaaa61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:54.742760 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9b6358-f485-41d6-b5ae-66d21ceaaa61 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:54.762656 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-8b9b6358-f485-41d6-b5ae-66d21ceaaa61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:05:54.762656 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:54.762656 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:54.762656 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:54.762656 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:54.762656 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:54.762656 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:54.762656 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:54.762656 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:54.762656 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:54.762656 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:54.762656 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:54.762656 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:05:54.762656 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:05:54.762656 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:05:54.763756 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:05:54.763756 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:54.763756 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:54.763756 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:54.763756 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:54.763756 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:54.763756 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:54.763756 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:54.763756 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:54.763756 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:54.763756 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:54.763756 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:54.763756 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:54.763756 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:54.763756 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:54.763756 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:54.763756 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:54.764801 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:05:54.764801 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:05:54.764801 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:05:54.764801 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:05:54.764801 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 255/494] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:05:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:54.769703 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f01bb75-82b7-47a7-84a3-623fb9060d02 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:54.769992 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f01bb75-82b7-47a7-84a3-623fb9060d02 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:54.770275 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f01bb75-82b7-47a7-84a3-623fb9060d02 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:54.770736 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:54.771855 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 256/495] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:05:54.772588 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f01bb75-82b7-47a7-84a3-623fb9060d02 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:54.791893 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98552409-171c-49a9-9c0d-89e1cdac9a75 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:54.829456 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f543fb51-971c-4cf1-adcb-46206f3d4d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:54.830235 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f543fb51-971c-4cf1-adcb-46206f3d4d65 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:54.830519 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f543fb51-971c-4cf1-adcb-46206f3d4d65 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:54.830798 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f543fb51-971c-4cf1-adcb-46206f3d4d65 None admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:54.831664 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f543fb51-971c-4cf1-adcb-46206f3d4d65 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:54.831973 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f543fb51-971c-4cf1-adcb-46206f3d4d65 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:54.833519 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f543fb51-971c-4cf1-adcb-46206f3d4d65 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:54.878096 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 240/496] 10.222.0.23 () {56 vars in 1133 bytes} [Tue Jun 21 18:05:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:54.878096 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f543fb51-971c-4cf1-adcb-46206f3d4d65 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:54.887239 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:54.898166 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f01bb75-82b7-47a7-84a3-623fb9060d02 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:54.936982 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4ee991-4c68-4b23-a191-9c8103fa822f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:54.937893 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ee991-4c68-4b23-a191-9c8103fa822f None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:54.938321 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ee991-4c68-4b23-a191-9c8103fa822f None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:54.938662 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ee991-4c68-4b23-a191-9c8103fa822f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:54.949702 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4ee991-4c68-4b23-a191-9c8103fa822f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:05:54.957692 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-af4ee991-4c68-4b23-a191-9c8103fa822f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:05:54.957692 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:54.957692 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:54.957692 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:54.957692 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:54.957692 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:54.957692 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:54.957692 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:54.957692 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:54.957692 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:54.957692 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:54.957692 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:54.957692 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:05:54.957692 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:05:54.957692 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:05:54.957692 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:05:54.958840 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:05:54.958840 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:05:54.958840 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:05:54.958840 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:05:54.958840 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:54.958840 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:54.958840 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:54.958840 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:54.958840 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:54.958840 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:54.958840 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:54.958840 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:54.958840 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:54.958840 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:54.958840 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:54.958840 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:54.959843 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:54.959843 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:54.959843 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:54.959843 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:54.959843 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:05:54.959843 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:05:54.959843 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:05:54.959843 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:05:54.959843 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:05:54.959843 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:05:54.959843 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:05:54.959843 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:05:54.966423 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 257/497] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jun 21 18:05:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:54.966423 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cd7bd6-305e-4d62-a732-0ce70ea30270 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:54.966423 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cd7bd6-305e-4d62-a732-0ce70ea30270 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:54.966423 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cd7bd6-305e-4d62-a732-0ce70ea30270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:54.982669 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-79cd7bd6-305e-4d62-a732-0ce70ea30270 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:54.983947 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-79cd7bd6-305e-4d62-a732-0ce70ea30270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:54.995113 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f543fb51-971c-4cf1-adcb-46206f3d4d65 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:55.030577 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad1730f-d561-46ae-aa70-8ab12e64ab96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:55.031287 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1730f-d561-46ae-aa70-8ab12e64ab96 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:55.031557 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1730f-d561-46ae-aa70-8ab12e64ab96 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:55.031905 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1730f-d561-46ae-aa70-8ab12e64ab96 None admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:55.032704 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad1730f-d561-46ae-aa70-8ab12e64ab96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:55.032991 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad1730f-d561-46ae-aa70-8ab12e64ab96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:55.034382 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad1730f-d561-46ae-aa70-8ab12e64ab96 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:55.047226 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 241/498] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:05:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:55.047977 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad1730f-d561-46ae-aa70-8ab12e64ab96 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:55.053713 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:55.060686 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 258/499] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:05:55.061643 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cd7bd6-305e-4d62-a732-0ce70ea30270 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:55.144272 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:55.151069 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad1730f-d561-46ae-aa70-8ab12e64ab96 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:55.161729 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cd7bd6-305e-4d62-a732-0ce70ea30270 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:55.185042 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ea97ba-572a-4d2f-82e8-5e1dbfccbdb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:55.185690 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea97ba-572a-4d2f-82e8-5e1dbfccbdb3 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:55.185977 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea97ba-572a-4d2f-82e8-5e1dbfccbdb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:55.186256 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea97ba-572a-4d2f-82e8-5e1dbfccbdb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:55.197444 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667a2c24-c03b-40ec-b37c-ba80da6ee9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:55.197899 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667a2c24-c03b-40ec-b37c-ba80da6ee9d0 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:55.198250 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667a2c24-c03b-40ec-b37c-ba80da6ee9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:55.198462 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667a2c24-c03b-40ec-b37c-ba80da6ee9d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:55.199217 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667a2c24-c03b-40ec-b37c-ba80da6ee9d0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:55.199526 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667a2c24-c03b-40ec-b37c-ba80da6ee9d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:55.200944 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667a2c24-c03b-40ec-b37c-ba80da6ee9d0 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:55.209043 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 259/500] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jun 21 18:05:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:55.210276 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f32d83-f2fe-4be2-8c6d-60eeabaeaac8 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:55.210564 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f32d83-f2fe-4be2-8c6d-60eeabaeaac8 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:55.210841 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f32d83-f2fe-4be2-8c6d-60eeabaeaac8 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:55.212059 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 260/501] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:05:55.213177 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f32d83-f2fe-4be2-8c6d-60eeabaeaac8 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:55.259909 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ea97ba-572a-4d2f-82e8-5e1dbfccbdb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:55.261429 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ea97ba-572a-4d2f-82e8-5e1dbfccbdb3 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:55.263390 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:55.318859 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f32d83-f2fe-4be2-8c6d-60eeabaeaac8 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:55.353928 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353d067a-60c5-4e8d-97aa-31e5b9fc63ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:55.354723 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d067a-60c5-4e8d-97aa-31e5b9fc63ed None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:55.355012 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d067a-60c5-4e8d-97aa-31e5b9fc63ed None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:55.355280 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d067a-60c5-4e8d-97aa-31e5b9fc63ed None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:55.362641 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 242/502] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:05:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:05:55.363035 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353d067a-60c5-4e8d-97aa-31e5b9fc63ed None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:55.363487 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ea97ba-572a-4d2f-82e8-5e1dbfccbdb3 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:55.364513 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353d067a-60c5-4e8d-97aa-31e5b9fc63ed None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:55.380685 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-353d067a-60c5-4e8d-97aa-31e5b9fc63ed None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 21 18:05:55.380685 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:55.380685 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:55.380685 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:55.380685 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:55.380685 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:55.380685 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:55.380685 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:55.380685 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:55.380685 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:55.380685 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:55.380685 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:55.380685 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:05:55.380685 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:05:55.380685 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:05:55.380685 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:05:55.381829 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:55.381829 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:55.381829 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:05:55.381829 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:05:55.381829 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:05:55.381829 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:05:55.381829 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:55.381829 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:55.381829 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:55.381829 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:55.381829 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:55.381829 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:55.381829 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:55.381829 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:55.381829 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:55.381829 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:55.381829 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:55.381829 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:55.382969 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:55.382969 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:55.382969 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:05:55.382969 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:05:55.382969 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:05:55.382969 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:05:55.382969 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:05:55.382969 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:05:55.382969 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 21 18:05:55.382969 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:05:55.382969 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 261/503] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:05:55 2022] GET /identity/v3/users/cinder => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:55.382969 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353d067a-60c5-4e8d-97aa-31e5b9fc63ed None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:55.387188 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:55.467948 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ea97ba-572a-4d2f-82e8-5e1dbfccbdb3 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:55.469627 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:55.490825 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353d067a-60c5-4e8d-97aa-31e5b9fc63ed None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:55.503157 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041f379b-ed41-409a-a5c7-deb5626d0cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:55.503979 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f379b-ed41-409a-a5c7-deb5626d0cbb None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:55.504250 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f379b-ed41-409a-a5c7-deb5626d0cbb None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:55.504561 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f379b-ed41-409a-a5c7-deb5626d0cbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:55.515266 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041f379b-ed41-409a-a5c7-deb5626d0cbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:55.516780 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041f379b-ed41-409a-a5c7-deb5626d0cbb None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:55.526178 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03484404-bf39-4368-97e3-995251d309ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:55.526984 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03484404-bf39-4368-97e3-995251d309ef None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:55.527263 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03484404-bf39-4368-97e3-995251d309ef None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:55.527541 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03484404-bf39-4368-97e3-995251d309ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:55.534730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-041f379b-ed41-409a-a5c7-deb5626d0cbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:05:55.534730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:55.534730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:55.534730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:55.534730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:55.534730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:55.534730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:55.534730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:55.534730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:55.534730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:55.534730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:55.534730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:55.534730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:05:55.534730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:05:55.534730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:05:55.535846 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:05:55.535846 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:55.535846 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:55.535846 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:55.535846 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:55.535846 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:55.535846 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:55.535846 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:55.535846 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:55.535846 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:55.535846 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:55.535846 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:55.535846 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:55.535846 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:55.535846 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:55.535846 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:55.535846 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:55.537139 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:05:55.537139 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:05:55.537139 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:05:55.537139 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:05:55.539531 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03484404-bf39-4368-97e3-995251d309ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:55.540593 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 243/504] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:05:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:55.542208 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03484404-bf39-4368-97e3-995251d309ef None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:55.542623 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc61715-4657-4307-8f1e-f0e513eeba1b None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:55.542918 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc61715-4657-4307-8f1e-f0e513eeba1b None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:55.543198 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc61715-4657-4307-8f1e-f0e513eeba1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:55.559537 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-3fc61715-4657-4307-8f1e-f0e513eeba1b None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:55.560691 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-03484404-bf39-4368-97e3-995251d309ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:05:55.560691 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:55.560691 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:55.560691 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:55.560691 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:55.560691 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:55.560691 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:55.560691 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:55.560691 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:55.560691 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:55.560691 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:55.560691 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:55.560691 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:05:55.560691 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:05:55.560691 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:05:55.561709 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:05:55.561709 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:55.561709 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:55.561709 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:55.561709 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:55.561709 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:55.561709 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:55.561709 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:55.561709 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:55.561709 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:55.561709 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:55.561709 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:55.561709 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:55.561709 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:55.561709 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:55.561709 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:55.561709 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:55.562832 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:05:55.562832 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:05:55.562832 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:05:55.562832 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:05:55.563068 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-3fc61715-4657-4307-8f1e-f0e513eeba1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:55.564531 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 262/505] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:05:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:55.565500 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03484404-bf39-4368-97e3-995251d309ef None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:55.622650 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:55.628591 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 244/506] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:05:55.629508 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc61715-4657-4307-8f1e-f0e513eeba1b None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:55.671047 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03484404-bf39-4368-97e3-995251d309ef None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:55.708002 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033c483a-884e-44ba-b510-6a3214240194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:55.708826 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033c483a-884e-44ba-b510-6a3214240194 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:55.709105 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033c483a-884e-44ba-b510-6a3214240194 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:55.709382 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033c483a-884e-44ba-b510-6a3214240194 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:55.716370 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:55.719773 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033c483a-884e-44ba-b510-6a3214240194 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:55.721265 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033c483a-884e-44ba-b510-6a3214240194 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:55.732577 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc61715-4657-4307-8f1e-f0e513eeba1b None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:55.738645 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-033c483a-884e-44ba-b510-6a3214240194 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:05:55.738645 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:55.738645 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:55.738645 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:55.738645 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:55.738645 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:55.738645 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:55.738645 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:55.738645 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:55.738645 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:55.738645 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:55.738645 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:55.738645 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:05:55.738645 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:05:55.738645 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:05:55.739665 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:05:55.739665 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:55.739665 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:55.739665 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:55.739665 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:55.739665 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:55.739665 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:55.739665 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:55.739665 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:55.739665 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:55.739665 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:55.739665 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:55.739665 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:55.739665 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:55.739665 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:55.739665 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:55.739665 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:55.740626 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:05:55.740626 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:05:55.740626 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:05:55.740626 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:05:55.740626 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 263/507] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:05:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:55.740980 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033c483a-884e-44ba-b510-6a3214240194 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:55.773438 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707c9c45-1116-419e-a7e3-67c7ae09f41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:55.774114 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c9c45-1116-419e-a7e3-67c7ae09f41f None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:55.774400 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c9c45-1116-419e-a7e3-67c7ae09f41f None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:55.774677 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c9c45-1116-419e-a7e3-67c7ae09f41f None admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:55.775559 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707c9c45-1116-419e-a7e3-67c7ae09f41f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:55.775870 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707c9c45-1116-419e-a7e3-67c7ae09f41f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:55.777342 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707c9c45-1116-419e-a7e3-67c7ae09f41f None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:55.795258 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 245/508] 10.222.0.23 () {56 vars in 1137 bytes} [Tue Jun 21 18:05:55 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:55.796095 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707c9c45-1116-419e-a7e3-67c7ae09f41f None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:55.842424 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033c483a-884e-44ba-b510-6a3214240194 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:55.851331 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:55.877558 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595dc4b2-8a72-4293-85f4-1e1dac631b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:55.878059 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595dc4b2-8a72-4293-85f4-1e1dac631b03 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:55.878341 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595dc4b2-8a72-4293-85f4-1e1dac631b03 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:55.878618 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595dc4b2-8a72-4293-85f4-1e1dac631b03 None admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:55.879345 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595dc4b2-8a72-4293-85f4-1e1dac631b03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:55.879651 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595dc4b2-8a72-4293-85f4-1e1dac631b03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:55.883517 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595dc4b2-8a72-4293-85f4-1e1dac631b03 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:55.894515 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 264/509] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:05:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:55.895359 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595dc4b2-8a72-4293-85f4-1e1dac631b03 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:55.901789 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707c9c45-1116-419e-a7e3-67c7ae09f41f None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:55.937213 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e35059-099e-4307-a4ca-654b9f94e48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:55.937879 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e35059-099e-4307-a4ca-654b9f94e48e None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:55.938166 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e35059-099e-4307-a4ca-654b9f94e48e None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:55.938449 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e35059-099e-4307-a4ca-654b9f94e48e None admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:55.939170 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e35059-099e-4307-a4ca-654b9f94e48e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:55.939477 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e35059-099e-4307-a4ca-654b9f94e48e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:55.940908 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e35059-099e-4307-a4ca-654b9f94e48e None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:55.952152 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 246/510] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:05:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:55.956875 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e35059-099e-4307-a4ca-654b9f94e48e None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:55.957328 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:55.996623 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595dc4b2-8a72-4293-85f4-1e1dac631b03 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:56.032150 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b7cad3-6279-4dee-82fd-4bbfaa0266d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:56.032980 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b7cad3-6279-4dee-82fd-4bbfaa0266d6 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:56.033263 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b7cad3-6279-4dee-82fd-4bbfaa0266d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:56.033537 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b7cad3-6279-4dee-82fd-4bbfaa0266d6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:56.045337 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b7cad3-6279-4dee-82fd-4bbfaa0266d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:05:56.052978 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-72b7cad3-6279-4dee-82fd-4bbfaa0266d6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:05:56.052978 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:56.052978 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:56.052978 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:56.052978 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:56.052978 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:56.052978 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:56.052978 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:56.052978 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:56.052978 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:56.052978 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:56.052978 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:56.052978 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:05:56.052978 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:05:56.052978 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:05:56.052978 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:05:56.053972 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:05:56.053972 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:05:56.053972 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:05:56.053972 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:05:56.053972 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:56.053972 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:56.053972 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:56.053972 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:56.053972 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:56.053972 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:56.053972 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:56.053972 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:56.053972 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:56.053972 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:56.053972 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:56.053972 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:56.054861 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:56.054861 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:56.054861 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:56.054861 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:56.054861 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:05:56.054861 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:05:56.054861 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:05:56.054861 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:05:56.054861 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:05:56.054861 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:05:56.054861 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:05:56.054861 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:05:56.054861 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 265/511] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jun 21 18:05:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:56.056043 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b7cad3-6279-4dee-82fd-4bbfaa0266d6 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:56.061118 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e35059-099e-4307-a4ca-654b9f94e48e None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:56.083589 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:56.096723 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8a6d60-e41b-4965-9fe9-c5c9aaf6c1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:56.097392 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8a6d60-e41b-4965-9fe9-c5c9aaf6c1eb None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:56.098155 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8a6d60-e41b-4965-9fe9-c5c9aaf6c1eb None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:56.098291 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8a6d60-e41b-4965-9fe9-c5c9aaf6c1eb None admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:56.098922 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8a6d60-e41b-4965-9fe9-c5c9aaf6c1eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:56.099229 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8a6d60-e41b-4965-9fe9-c5c9aaf6c1eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:56.100671 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8a6d60-e41b-4965-9fe9-c5c9aaf6c1eb None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:56.113160 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 247/512] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:05:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:56.113947 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8a6d60-e41b-4965-9fe9-c5c9aaf6c1eb None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:56.161224 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b7cad3-6279-4dee-82fd-4bbfaa0266d6 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:56.174840 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:56.194921 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41e45a9-2d36-4ebe-ba1e-de80dd0eaf69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:56.195578 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41e45a9-2d36-4ebe-ba1e-de80dd0eaf69 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:56.195865 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41e45a9-2d36-4ebe-ba1e-de80dd0eaf69 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:56.196151 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41e45a9-2d36-4ebe-ba1e-de80dd0eaf69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:56.214063 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8a6d60-e41b-4965-9fe9-c5c9aaf6c1eb None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:56.248988 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ead22c2-586e-447a-a6d7-64b460f11112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:56.249661 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ead22c2-586e-447a-a6d7-64b460f11112 None admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:56.249941 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ead22c2-586e-447a-a6d7-64b460f11112 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:56.250216 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ead22c2-586e-447a-a6d7-64b460f11112 None admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:56.250919 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ead22c2-586e-447a-a6d7-64b460f11112 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:56.252351 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ead22c2-586e-447a-a6d7-64b460f11112 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:56.267157 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41e45a9-2d36-4ebe-ba1e-de80dd0eaf69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:56.268661 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41e45a9-2d36-4ebe-ba1e-de80dd0eaf69 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:56.295879 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:56.324270 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 248/513] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Jun 21 18:05:56 2022] POST /identity/v3/users => generated 247 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:05:56.325174 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ead22c2-586e-447a-a6d7-64b460f11112 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:56.376512 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 266/514] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:05:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:05:56.378157 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41e45a9-2d36-4ebe-ba1e-de80dd0eaf69 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:56.454261 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ead22c2-586e-447a-a6d7-64b460f11112 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:56.474123 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:56.495849 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5467f5c-5c84-4905-a9dc-463a56e355cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:56.496869 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41e45a9-2d36-4ebe-ba1e-de80dd0eaf69 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:56.496980 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5467f5c-5c84-4905-a9dc-463a56e355cb None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:56.496980 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5467f5c-5c84-4905-a9dc-463a56e355cb None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:56.497246 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5467f5c-5c84-4905-a9dc-463a56e355cb None admin] PATH_INFO: `/v3/users/glance` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:56.506941 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5467f5c-5c84-4905-a9dc-463a56e355cb None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:56.508420 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5467f5c-5c84-4905-a9dc-463a56e355cb None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:56.524514 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-e5467f5c-5c84-4905-a9dc-463a56e355cb None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 21 18:05:56.524514 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:56.524514 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:56.524514 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:56.524514 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:56.524514 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:56.524514 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:56.524514 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:56.524514 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:56.524514 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:56.524514 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:56.524514 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:56.524514 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:05:56.524514 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:05:56.524514 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:05:56.524514 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:05:56.526101 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:56.526101 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:56.526101 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:05:56.526101 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:05:56.526101 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:05:56.526101 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:05:56.526101 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:56.526101 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:56.526101 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:56.526101 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:56.526101 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:56.526101 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:56.526101 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:56.526101 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:56.526101 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:56.526101 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:56.526101 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:56.526101 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:56.527166 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:56.527166 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:56.527166 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:05:56.527166 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:05:56.527166 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:05:56.527166 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:05:56.527166 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:05:56.527166 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:05:56.527166 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 21 18:05:56.527166 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:05:56.527166 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 249/515] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:05:56 2022] GET /identity/v3/users/glance => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:56.527739 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5467f5c-5c84-4905-a9dc-463a56e355cb None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:56.528644 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:56.546792 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7661d49-cef8-4089-b82a-ec7bf5b8daa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:56.550338 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7661d49-cef8-4089-b82a-ec7bf5b8daa1 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:56.550338 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7661d49-cef8-4089-b82a-ec7bf5b8daa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:56.550338 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7661d49-cef8-4089-b82a-ec7bf5b8daa1 None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:56.550338 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7661d49-cef8-4089-b82a-ec7bf5b8daa1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:56.550338 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7661d49-cef8-4089-b82a-ec7bf5b8daa1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:56.550733 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7661d49-cef8-4089-b82a-ec7bf5b8daa1 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:56.562704 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 267/516] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jun 21 18:05:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:56.563742 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7661d49-cef8-4089-b82a-ec7bf5b8daa1 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:56.636835 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5467f5c-5c84-4905-a9dc-463a56e355cb None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:56.671496 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7661d49-cef8-4089-b82a-ec7bf5b8daa1 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:56.672971 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b7a4f-fe01-41af-80c1-f9a411e9d31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:56.673787 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b7a4f-fe01-41af-80c1-f9a411e9d31b None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:56.674079 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b7a4f-fe01-41af-80c1-f9a411e9d31b None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:56.674353 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b7a4f-fe01-41af-80c1-f9a411e9d31b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:56.681342 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2b7a4f-fe01-41af-80c1-f9a411e9d31b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:56.682909 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2b7a4f-fe01-41af-80c1-f9a411e9d31b None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:56.698612 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-8e2b7a4f-fe01-41af-80c1-f9a411e9d31b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:05:56.698612 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:56.698612 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:56.698612 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:56.698612 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:56.698612 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:56.698612 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:56.698612 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:56.698612 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:56.698612 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:56.698612 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:56.698612 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:56.698612 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:05:56.698612 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:05:56.698612 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:05:56.699540 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:05:56.699540 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:56.699540 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:56.699540 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:56.699540 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:56.699540 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:56.699540 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:56.699540 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:56.699540 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:56.699540 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:56.699540 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:56.699540 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:56.699540 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:56.699540 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:56.699540 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:56.699540 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:56.699540 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:56.700449 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:05:56.700449 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:05:56.700449 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:05:56.700449 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:05:56.700449 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:05:56.700449 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:05:56.700449 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:05:56.700449 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:05:56.700449 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 250/517] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jun 21 18:05:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:56.700941 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b7a4f-fe01-41af-80c1-f9a411e9d31b None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:56.714472 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf81f67-b04c-45d1-a986-19fd54ca498d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:56.715190 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf81f67-b04c-45d1-a986-19fd54ca498d None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:56.715469 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf81f67-b04c-45d1-a986-19fd54ca498d None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:56.715741 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf81f67-b04c-45d1-a986-19fd54ca498d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:56.726896 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf81f67-b04c-45d1-a986-19fd54ca498d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:56.728346 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf81f67-b04c-45d1-a986-19fd54ca498d None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:56.746544 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-8cf81f67-b04c-45d1-a986-19fd54ca498d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:05:56.746544 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:56.746544 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:56.746544 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:56.746544 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:56.746544 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:56.746544 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:56.746544 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:56.746544 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:56.746544 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:56.746544 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:56.746544 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:56.746544 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:05:56.746544 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:05:56.746544 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:05:56.747701 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:05:56.747701 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:56.747701 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:56.747701 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:56.747701 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:56.747701 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:56.747701 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:56.747701 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:56.747701 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:56.747701 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:56.747701 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:56.747701 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:56.747701 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:56.747701 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:56.747701 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:56.747701 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:56.747701 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:56.748801 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:05:56.748801 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:05:56.748801 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:05:56.748801 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:05:56.748801 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 268/518] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:05:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:56.750665 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:56.753114 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf81f67-b04c-45d1-a986-19fd54ca498d None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:56.802880 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:56.821547 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b7a4f-fe01-41af-80c1-f9a411e9d31b None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:56.862557 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316ad8a0-aa35-44fe-ad1a-89175bab77b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:56.863257 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ad8a0-aa35-44fe-ad1a-89175bab77b6 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:56.863560 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ad8a0-aa35-44fe-ad1a-89175bab77b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:56.863810 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ad8a0-aa35-44fe-ad1a-89175bab77b6 None admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:56.864722 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316ad8a0-aa35-44fe-ad1a-89175bab77b6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:56.865030 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316ad8a0-aa35-44fe-ad1a-89175bab77b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:56.866514 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316ad8a0-aa35-44fe-ad1a-89175bab77b6 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:56.874770 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf81f67-b04c-45d1-a986-19fd54ca498d None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:56.893896 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 251/519] 10.222.0.23 () {56 vars in 1137 bytes} [Tue Jun 21 18:05:56 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:56.893896 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316ad8a0-aa35-44fe-ad1a-89175bab77b6 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:56.921240 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202d8b35-378b-4837-8e54-1ed2ff6e5e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:56.922159 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202d8b35-378b-4837-8e54-1ed2ff6e5e2d None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:56.923332 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202d8b35-378b-4837-8e54-1ed2ff6e5e2d None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:56.923689 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202d8b35-378b-4837-8e54-1ed2ff6e5e2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:56.937423 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202d8b35-378b-4837-8e54-1ed2ff6e5e2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:56.939041 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202d8b35-378b-4837-8e54-1ed2ff6e5e2d None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:56.958656 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-202d8b35-378b-4837-8e54-1ed2ff6e5e2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:05:56.958656 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:56.958656 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:56.958656 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:56.958656 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:56.958656 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:56.958656 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:56.958656 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:56.958656 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:56.958656 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:56.958656 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:56.958656 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:56.958656 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:05:56.958656 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:05:56.958656 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:05:56.959787 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:05:56.959787 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:56.959787 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:56.959787 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:56.959787 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:56.959787 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:56.959787 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:56.959787 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:56.959787 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:56.959787 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:56.959787 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:56.959787 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:56.959787 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:56.959787 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:56.959787 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:56.959787 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:56.959787 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:56.960912 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:05:56.960912 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:05:56.960912 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:05:56.960912 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:05:56.961508 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 269/520] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:05:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:56.962221 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202d8b35-378b-4837-8e54-1ed2ff6e5e2d None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:57.013052 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:57.019086 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316ad8a0-aa35-44fe-ad1a-89175bab77b6 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:57.061821 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0704064a-6d19-4038-8e6f-33357ad98a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:57.062502 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0704064a-6d19-4038-8e6f-33357ad98a94 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:57.062788 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0704064a-6d19-4038-8e6f-33357ad98a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:57.063049 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0704064a-6d19-4038-8e6f-33357ad98a94 None admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:57.063851 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0704064a-6d19-4038-8e6f-33357ad98a94 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:57.064162 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0704064a-6d19-4038-8e6f-33357ad98a94 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:57.065683 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0704064a-6d19-4038-8e6f-33357ad98a94 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:57.077160 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:57.091729 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 252/521] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jun 21 18:05:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:57.092962 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202d8b35-378b-4837-8e54-1ed2ff6e5e2d None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:57.093088 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0704064a-6d19-4038-8e6f-33357ad98a94 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:57.128672 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f6d95d-2c98-4f66-a82a-0f5db96f7c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:57.129399 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6d95d-2c98-4f66-a82a-0f5db96f7c09 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:57.129685 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6d95d-2c98-4f66-a82a-0f5db96f7c09 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:57.129960 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6d95d-2c98-4f66-a82a-0f5db96f7c09 None admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:57.130692 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f6d95d-2c98-4f66-a82a-0f5db96f7c09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:57.130998 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f6d95d-2c98-4f66-a82a-0f5db96f7c09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:57.132407 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f6d95d-2c98-4f66-a82a-0f5db96f7c09 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:57.151083 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 270/522] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:05:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:57.151083 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f6d95d-2c98-4f66-a82a-0f5db96f7c09 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:57.199166 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0704064a-6d19-4038-8e6f-33357ad98a94 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:57.236297 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86acc1fc-7051-4b6c-bdc2-3ed443caeb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:57.236982 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86acc1fc-7051-4b6c-bdc2-3ed443caeb25 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:57.237267 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86acc1fc-7051-4b6c-bdc2-3ed443caeb25 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:57.237545 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86acc1fc-7051-4b6c-bdc2-3ed443caeb25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:57.261018 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f6d95d-2c98-4f66-a82a-0f5db96f7c09 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:57.264987 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:57.298128 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88305379-f14b-4706-8d7f-a3c5a5dc13ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:57.298795 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88305379-f14b-4706-8d7f-a3c5a5dc13ba None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:57.299232 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88305379-f14b-4706-8d7f-a3c5a5dc13ba None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:57.299509 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88305379-f14b-4706-8d7f-a3c5a5dc13ba None admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:57.300703 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88305379-f14b-4706-8d7f-a3c5a5dc13ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:57.301127 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88305379-f14b-4706-8d7f-a3c5a5dc13ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:57.302958 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88305379-f14b-4706-8d7f-a3c5a5dc13ba None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:57.309217 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86acc1fc-7051-4b6c-bdc2-3ed443caeb25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:57.310679 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86acc1fc-7051-4b6c-bdc2-3ed443caeb25 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:57.316185 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 271/523] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:05:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:57.316846 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88305379-f14b-4706-8d7f-a3c5a5dc13ba None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:57.334096 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:57.423160 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88305379-f14b-4706-8d7f-a3c5a5dc13ba None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:57.423801 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 253/524] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:05:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:05:57.424877 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86acc1fc-7051-4b6c-bdc2-3ed443caeb25 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:57.464991 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a4067-a174-47e5-9f74-e860e4a175dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:57.465653 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a4067-a174-47e5-9f74-e860e4a175dc None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:57.465960 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a4067-a174-47e5-9f74-e860e4a175dc None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:57.466205 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a4067-a174-47e5-9f74-e860e4a175dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:57.467041 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418a4067-a174-47e5-9f74-e860e4a175dc None admin] RBAC: Adding query filter params (role.id=acf4e26e0ceb41ddad002dd4edf6373c, scope.project.id=2a4699d1ec70456da5f8e0aa4b1439b7, user.id=904255dd3c1f406eb10f6fc65f9cd098) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:57.467354 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418a4067-a174-47e5-9f74-e860e4a175dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:57.468842 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418a4067-a174-47e5-9f74-e860e4a175dc None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:57.474758 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 272/525] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jun 21 18:05:57 2022] GET /identity/v3/role_assignments?user.id=904255dd3c1f406eb10f6fc65f9cd098&scope.project.id=2a4699d1ec70456da5f8e0aa4b1439b7&role.id=acf4e26e0ceb41ddad002dd4edf6373c => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:57.475561 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a4067-a174-47e5-9f74-e860e4a175dc None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:57.504215 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:57.559448 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86acc1fc-7051-4b6c-bdc2-3ed443caeb25 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:57.596918 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:57.612902 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52673a43-7b92-4e09-8910-cd7b3f37b2df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:57.614568 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52673a43-7b92-4e09-8910-cd7b3f37b2df None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:57.615126 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52673a43-7b92-4e09-8910-cd7b3f37b2df None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:57.617010 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52673a43-7b92-4e09-8910-cd7b3f37b2df None admin] PATH_INFO: `/v3/projects/service` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:57.624988 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a4067-a174-47e5-9f74-e860e4a175dc None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:57.627726 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52673a43-7b92-4e09-8910-cd7b3f37b2df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:57.634992 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52673a43-7b92-4e09-8910-cd7b3f37b2df None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:57.653067 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-52673a43-7b92-4e09-8910-cd7b3f37b2df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:05:57.653067 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:57.653067 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:57.653067 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:57.653067 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:57.653067 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:57.653067 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:57.653067 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:57.653067 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:57.653067 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:57.653067 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:57.653067 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:57.653067 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:05:57.653067 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:05:57.653067 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:05:57.653989 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:05:57.653989 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:57.653989 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:57.653989 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:57.653989 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:57.653989 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:57.653989 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:57.653989 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:57.653989 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:57.653989 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:57.653989 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:57.653989 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:57.653989 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:57.653989 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:57.653989 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:57.653989 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:57.653989 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:57.655099 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:05:57.655099 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:05:57.655099 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:05:57.655099 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:05:57.655099 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:05:57.655099 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:05:57.655099 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:05:57.655099 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:05:57.657021 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 254/526] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jun 21 18:05:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:57.657964 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52673a43-7b92-4e09-8910-cd7b3f37b2df None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:57.674101 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f426d9-0b97-42ff-ba5d-4346a98ecc62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:57.675008 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f426d9-0b97-42ff-ba5d-4346a98ecc62 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:57.675298 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f426d9-0b97-42ff-ba5d-4346a98ecc62 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:57.675584 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f426d9-0b97-42ff-ba5d-4346a98ecc62 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:57.686209 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f426d9-0b97-42ff-ba5d-4346a98ecc62 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:57.687693 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f426d9-0b97-42ff-ba5d-4346a98ecc62 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:57.704225 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-d8f426d9-0b97-42ff-ba5d-4346a98ecc62 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 21 18:05:57.704225 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:57.704225 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:57.704225 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:57.704225 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:57.704225 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:57.704225 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:57.704225 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:57.704225 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:57.704225 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:57.704225 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:57.704225 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:57.704225 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:05:57.704225 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:05:57.704225 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:05:57.704225 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:05:57.707521 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:57.707521 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:57.707521 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:05:57.707521 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:05:57.707521 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:05:57.707521 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:05:57.707521 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:57.707521 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:57.707521 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:57.707521 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:57.707521 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:57.707521 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:57.707521 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:57.707521 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:57.707521 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:57.707521 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:57.707521 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:57.707521 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:57.708697 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:57.708697 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:57.708697 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:05:57.708697 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:05:57.708697 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:05:57.708697 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:05:57.708697 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:05:57.708697 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:05:57.708697 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 21 18:05:57.708697 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:05:57.710603 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 273/527] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jun 21 18:05:57 2022] GET /identity/v3/users/neutron => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:57.711435 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f426d9-0b97-42ff-ba5d-4346a98ecc62 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:57.774586 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52673a43-7b92-4e09-8910-cd7b3f37b2df None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:57.792144 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:57.811997 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd87103-2ec6-4260-866c-89319d46abab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:57.813222 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd87103-2ec6-4260-866c-89319d46abab None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:57.813652 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd87103-2ec6-4260-866c-89319d46abab None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:57.814072 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd87103-2ec6-4260-866c-89319d46abab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:57.822460 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f426d9-0b97-42ff-ba5d-4346a98ecc62 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:57.826951 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd87103-2ec6-4260-866c-89319d46abab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:57.828755 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd87103-2ec6-4260-866c-89319d46abab None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:57.849427 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-bfd87103-2ec6-4260-866c-89319d46abab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:05:57.849427 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:57.849427 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:57.849427 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:57.849427 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:57.849427 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:57.849427 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:57.849427 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:57.849427 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:57.849427 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:57.849427 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:57.849427 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:57.849427 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:05:57.849427 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:05:57.849427 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:05:57.850567 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:05:57.850567 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:57.850567 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:57.850567 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:57.850567 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:57.850567 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:57.850567 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:57.850567 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:57.850567 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:57.850567 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:57.850567 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:57.850567 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:57.850567 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:57.850567 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:57.850567 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:57.850567 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:57.850567 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:57.851640 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:05:57.851640 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:05:57.851640 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:05:57.851640 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:05:57.858955 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 255/528] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:05:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:57.860247 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd87103-2ec6-4260-866c-89319d46abab None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:57.871496 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2f860c-701f-40ef-82e7-5c4348ec8d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:57.872174 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f860c-701f-40ef-82e7-5c4348ec8d61 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:57.872486 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f860c-701f-40ef-82e7-5c4348ec8d61 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:57.872767 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f860c-701f-40ef-82e7-5c4348ec8d61 None admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:57.873627 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2f860c-701f-40ef-82e7-5c4348ec8d61 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:57.873987 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2f860c-701f-40ef-82e7-5c4348ec8d61 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:57.875445 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2f860c-701f-40ef-82e7-5c4348ec8d61 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:57.892789 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:57.901385 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 274/529] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jun 21 18:05:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:57.902237 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2f860c-701f-40ef-82e7-5c4348ec8d61 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:57.974070 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd87103-2ec6-4260-866c-89319d46abab None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:58.012535 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2f860c-701f-40ef-82e7-5c4348ec8d61 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:58.014306 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f4300d-65e2-40db-bfaf-f6d610c6e398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:58.015258 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4300d-65e2-40db-bfaf-f6d610c6e398 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:58.015726 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4300d-65e2-40db-bfaf-f6d610c6e398 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:58.016145 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4300d-65e2-40db-bfaf-f6d610c6e398 None admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:58.017161 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f4300d-65e2-40db-bfaf-f6d610c6e398 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:58.017610 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f4300d-65e2-40db-bfaf-f6d610c6e398 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:58.019181 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f4300d-65e2-40db-bfaf-f6d610c6e398 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:58.039488 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 256/530] 10.222.0.23 () {56 vars in 1139 bytes} [Tue Jun 21 18:05:57 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:58.040655 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f4300d-65e2-40db-bfaf-f6d610c6e398 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:58.055240 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b594569-6466-4152-ab9a-d4cfe7cd1b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:58.055965 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b594569-6466-4152-ab9a-d4cfe7cd1b2d None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:58.056313 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b594569-6466-4152-ab9a-d4cfe7cd1b2d None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:58.056710 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b594569-6466-4152-ab9a-d4cfe7cd1b2d None admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:58.057789 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b594569-6466-4152-ab9a-d4cfe7cd1b2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:58.058139 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b594569-6466-4152-ab9a-d4cfe7cd1b2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:58.059688 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b594569-6466-4152-ab9a-d4cfe7cd1b2d None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:58.071155 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:58.079401 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 275/531] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:05:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:58.079401 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b594569-6466-4152-ab9a-d4cfe7cd1b2d None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:58.144530 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:58.155835 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f4300d-65e2-40db-bfaf-f6d610c6e398 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:58.192235 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b594569-6466-4152-ab9a-d4cfe7cd1b2d None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:58.193412 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df1add-50dc-41e9-8386-916a4ccb06b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:58.194366 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df1add-50dc-41e9-8386-916a4ccb06b1 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:58.194865 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df1add-50dc-41e9-8386-916a4ccb06b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:58.195294 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df1add-50dc-41e9-8386-916a4ccb06b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:58.196261 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41df1add-50dc-41e9-8386-916a4ccb06b1 None admin] RBAC: Adding query filter params (role.id=acf4e26e0ceb41ddad002dd4edf6373c, scope.project.id=2a4699d1ec70456da5f8e0aa4b1439b7, user.id=77c6e43555944902a22ba2fe17a8e99f) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:58.197335 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41df1add-50dc-41e9-8386-916a4ccb06b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:58.199112 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41df1add-50dc-41e9-8386-916a4ccb06b1 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:58.205925 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 257/532] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jun 21 18:05:58 2022] GET /identity/v3/role_assignments?user.id=77c6e43555944902a22ba2fe17a8e99f&scope.project.id=2a4699d1ec70456da5f8e0aa4b1439b7&role.id=acf4e26e0ceb41ddad002dd4edf6373c => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:58.207019 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df1add-50dc-41e9-8386-916a4ccb06b1 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:58.256498 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0834e099-243b-4d9c-bf16-563130e1db98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:58.257305 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0834e099-243b-4d9c-bf16-563130e1db98 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:58.257849 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0834e099-243b-4d9c-bf16-563130e1db98 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:58.257966 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0834e099-243b-4d9c-bf16-563130e1db98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:58.318336 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:58.325061 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df1add-50dc-41e9-8386-916a4ccb06b1 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:58.364878 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc4779c-801c-47ab-ada1-919517ce225f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:58.366223 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc4779c-801c-47ab-ada1-919517ce225f None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:58.366763 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc4779c-801c-47ab-ada1-919517ce225f None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:58.367188 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc4779c-801c-47ab-ada1-919517ce225f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:58.375345 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc4779c-801c-47ab-ada1-919517ce225f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:58.377242 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc4779c-801c-47ab-ada1-919517ce225f None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:58.392321 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-fbc4779c-801c-47ab-ada1-919517ce225f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:05:58.392321 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:58.392321 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:58.392321 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:58.392321 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:58.392321 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:58.392321 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:58.392321 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:58.392321 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:58.392321 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:58.392321 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:58.392321 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:58.392321 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:05:58.392321 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:05:58.392321 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:05:58.393420 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:05:58.393420 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:58.393420 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:58.393420 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:58.393420 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:58.393420 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:58.393420 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:58.393420 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:58.393420 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:58.393420 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:58.393420 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:58.393420 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:58.393420 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:58.393420 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:58.393420 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:58.393420 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:58.393420 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:58.394294 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:05:58.394294 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:05:58.394294 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:05:58.394294 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:05:58.394294 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:05:58.394294 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:05:58.394294 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:05:58.394294 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:05:58.396625 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 258/533] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jun 21 18:05:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:58.403246 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc4779c-801c-47ab-ada1-919517ce225f None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:58.408281 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0834e099-243b-4d9c-bf16-563130e1db98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:58.409797 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0834e099-243b-4d9c-bf16-563130e1db98 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:58.479568 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:58.515804 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc4779c-801c-47ab-ada1-919517ce225f None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:58.530069 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 276/534] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:05:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:05:58.535356 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0834e099-243b-4d9c-bf16-563130e1db98 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:58.575692 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f29c2f-41dc-4c8e-b8e9-1041ffafe60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:58.577417 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f29c2f-41dc-4c8e-b8e9-1041ffafe60a None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:58.578028 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f29c2f-41dc-4c8e-b8e9-1041ffafe60a None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:58.579638 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f29c2f-41dc-4c8e-b8e9-1041ffafe60a None admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:58.580712 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f29c2f-41dc-4c8e-b8e9-1041ffafe60a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:58.581195 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f29c2f-41dc-4c8e-b8e9-1041ffafe60a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:58.582914 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f29c2f-41dc-4c8e-b8e9-1041ffafe60a None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:58.600013 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 259/535] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jun 21 18:05:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:05:58.606542 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ad652-fb59-4d43-9163-bea304de79b3 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:58.607469 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ad652-fb59-4d43-9163-bea304de79b3 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:58.607899 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ad652-fb59-4d43-9163-bea304de79b3 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:58.610162 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 260/536] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:58 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:05:58.612356 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832ad652-fb59-4d43-9163-bea304de79b3 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:58.630885 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:58.695398 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0834e099-243b-4d9c-bf16-563130e1db98 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:58.737833 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58ca358-3f09-4720-8179-69ede376566f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:58.738947 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ca358-3f09-4720-8179-69ede376566f None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:58.739236 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ca358-3f09-4720-8179-69ede376566f None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:58.739516 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ca358-3f09-4720-8179-69ede376566f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:58.753505 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58ca358-3f09-4720-8179-69ede376566f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:58.755248 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58ca358-3f09-4720-8179-69ede376566f None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:58.756912 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832ad652-fb59-4d43-9163-bea304de79b3 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:58.778804 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-b58ca358-3f09-4720-8179-69ede376566f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:05:58.778804 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:58.778804 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:58.778804 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:58.778804 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:58.778804 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:58.778804 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:58.778804 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:58.778804 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:58.778804 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:58.778804 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:58.778804 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:58.778804 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:05:58.778804 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:05:58.778804 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:05:58.779737 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:05:58.779737 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:58.779737 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:58.779737 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:58.779737 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:58.779737 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:58.779737 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:58.779737 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:58.779737 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:58.779737 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:58.779737 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:58.779737 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:58.779737 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:58.779737 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:58.779737 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:58.779737 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:58.779737 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:58.780642 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:05:58.780642 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:05:58.780642 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:05:58.780642 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:05:58.780642 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 277/537] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:05:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:05:58.781953 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc9c8f0-a28b-4b9c-9a9b-89b50c73e125 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:58.782207 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc9c8f0-a28b-4b9c-9a9b-89b50c73e125 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:58.782482 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc9c8f0-a28b-4b9c-9a9b-89b50c73e125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:58.796935 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a2247f-e71f-4e9a-bb40-4cd31e81a589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:58.797993 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a2247f-e71f-4e9a-bb40-4cd31e81a589 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:58.798447 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a2247f-e71f-4e9a-bb40-4cd31e81a589 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:58.798853 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a2247f-e71f-4e9a-bb40-4cd31e81a589 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:58.800111 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a2247f-e71f-4e9a-bb40-4cd31e81a589 None admin] RBAC: Adding query filter params (role.id=acf4e26e0ceb41ddad002dd4edf6373c, scope.project.id=2a4699d1ec70456da5f8e0aa4b1439b7, user.id=9672794bc1ca4dcd8259d51d9290384b) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:58.800621 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a2247f-e71f-4e9a-bb40-4cd31e81a589 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:58.802424 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a2247f-e71f-4e9a-bb40-4cd31e81a589 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:58.803262 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-edc9c8f0-a28b-4b9c-9a9b-89b50c73e125 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:58.804775 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-edc9c8f0-a28b-4b9c-9a9b-89b50c73e125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:05:58.808704 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:58.810501 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 261/538] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jun 21 18:05:58 2022] GET /identity/v3/role_assignments?user.id=9672794bc1ca4dcd8259d51d9290384b&scope.project.id=2a4699d1ec70456da5f8e0aa4b1439b7&role.id=acf4e26e0ceb41ddad002dd4edf6373c => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:58.811595 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a2247f-e71f-4e9a-bb40-4cd31e81a589 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:58.907992 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:58.920477 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 278/539] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:05:58.940316 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc9c8f0-a28b-4b9c-9a9b-89b50c73e125 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:58.944803 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a2247f-e71f-4e9a-bb40-4cd31e81a589 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:59.025123 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e785e892-44e7-4974-9739-4a64f71d440b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:59.025815 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e785e892-44e7-4974-9739-4a64f71d440b None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:59.026095 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e785e892-44e7-4974-9739-4a64f71d440b None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:59.026375 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e785e892-44e7-4974-9739-4a64f71d440b None admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:59.027119 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e785e892-44e7-4974-9739-4a64f71d440b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:59.027426 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e785e892-44e7-4974-9739-4a64f71d440b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:59.028835 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e785e892-44e7-4974-9739-4a64f71d440b None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:59.059808 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 262/540] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:05:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:59.065323 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e785e892-44e7-4974-9739-4a64f71d440b None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:59.114031 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc9c8f0-a28b-4b9c-9a9b-89b50c73e125 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:59.155495 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6e4e8d-1631-4b67-8e99-74e7ca7bea8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:59.156478 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e4e8d-1631-4b67-8e99-74e7ca7bea8f None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:59.156773 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e4e8d-1631-4b67-8e99-74e7ca7bea8f None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:59.157825 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e4e8d-1631-4b67-8e99-74e7ca7bea8f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:59.185406 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:59.191438 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6e4e8d-1631-4b67-8e99-74e7ca7bea8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:05:59.199451 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-1c6e4e8d-1631-4b67-8e99-74e7ca7bea8f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:05:59.199451 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:59.199451 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:59.199451 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:59.199451 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:59.199451 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:59.199451 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:59.199451 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:59.199451 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:59.199451 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:59.199451 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:59.199451 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:59.199451 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:05:59.199451 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:05:59.199451 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:05:59.199451 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:05:59.204614 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:05:59.204614 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:05:59.204614 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:05:59.204614 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:05:59.204614 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:59.204614 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:59.204614 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:59.204614 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:59.204614 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:59.204614 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:59.204614 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:59.204614 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:59.204614 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:59.204614 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:59.204614 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:59.204614 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:59.206250 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:59.206250 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:59.206250 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:59.206250 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:59.206250 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:05:59.206250 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:05:59.206250 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:05:59.206250 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:05:59.206250 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:05:59.206250 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:05:59.206250 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:05:59.206250 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:05:59.207480 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e785e892-44e7-4974-9739-4a64f71d440b None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:59.213451 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 279/541] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jun 21 18:05:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:05:59.221761 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6e4e8d-1631-4b67-8e99-74e7ca7bea8f None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:59.252172 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14049a11-ae49-4a5c-a91e-43cdad7296c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:59.253186 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14049a11-ae49-4a5c-a91e-43cdad7296c3 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:59.253485 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14049a11-ae49-4a5c-a91e-43cdad7296c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:59.253795 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14049a11-ae49-4a5c-a91e-43cdad7296c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:59.261263 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:59.267830 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14049a11-ae49-4a5c-a91e-43cdad7296c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:59.269983 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14049a11-ae49-4a5c-a91e-43cdad7296c3 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:59.293373 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-14049a11-ae49-4a5c-a91e-43cdad7296c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:05:59.293373 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:59.293373 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:59.293373 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:59.293373 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:59.293373 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:59.293373 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:59.293373 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:59.293373 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:59.293373 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:59.293373 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:59.293373 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:59.293373 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:05:59.293373 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:05:59.293373 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:05:59.294454 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:05:59.294454 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:59.294454 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:59.294454 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:59.294454 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:59.294454 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:59.294454 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:59.294454 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:59.294454 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:59.294454 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:59.294454 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:59.294454 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:59.294454 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:59.294454 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:59.294454 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:59.294454 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:59.294454 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:59.295375 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:05:59.295375 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:05:59.295375 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:05:59.295375 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:05:59.295375 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:05:59.295375 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:05:59.295375 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:05:59.295375 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:05:59.303979 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 263/542] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jun 21 18:05:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:59.310177 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14049a11-ae49-4a5c-a91e-43cdad7296c3 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:59.435652 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6e4e8d-1631-4b67-8e99-74e7ca7bea8f None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:59.478512 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14049a11-ae49-4a5c-a91e-43cdad7296c3 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:59.505143 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f721029-46ff-4e35-9f88-b96cd4cb6547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:59.505841 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f721029-46ff-4e35-9f88-b96cd4cb6547 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:59.506128 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f721029-46ff-4e35-9f88-b96cd4cb6547 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:59.506405 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f721029-46ff-4e35-9f88-b96cd4cb6547 None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:59.507168 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f721029-46ff-4e35-9f88-b96cd4cb6547 None admin] RBAC: Adding query filter params (name=service) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:59.507477 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f721029-46ff-4e35-9f88-b96cd4cb6547 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:59.516415 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f721029-46ff-4e35-9f88-b96cd4cb6547 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:59.520417 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 280/543] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jun 21 18:05:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:05:59.524625 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f721029-46ff-4e35-9f88-b96cd4cb6547 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:59.538715 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040e3108-156c-4089-b7b0-5d05a48e82c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:59.539381 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040e3108-156c-4089-b7b0-5d05a48e82c2 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:59.539666 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040e3108-156c-4089-b7b0-5d05a48e82c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:59.539941 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040e3108-156c-4089-b7b0-5d05a48e82c2 None admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:59.541261 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040e3108-156c-4089-b7b0-5d05a48e82c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:59.541626 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040e3108-156c-4089-b7b0-5d05a48e82c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:59.543074 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040e3108-156c-4089-b7b0-5d05a48e82c2 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:59.564606 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 264/544] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jun 21 18:05:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:59.570556 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040e3108-156c-4089-b7b0-5d05a48e82c2 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:59.653870 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:59.687666 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:05:59.691002 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f721029-46ff-4e35-9f88-b96cd4cb6547 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:59.746503 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f410ee2b-9cbb-45f3-939b-e0e4c6239af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:59.747711 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f410ee2b-9cbb-45f3-939b-e0e4c6239af8 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:59.749644 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f410ee2b-9cbb-45f3-939b-e0e4c6239af8 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:59.750129 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f410ee2b-9cbb-45f3-939b-e0e4c6239af8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:59.778806 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f410ee2b-9cbb-45f3-939b-e0e4c6239af8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:59.780118 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f410ee2b-9cbb-45f3-939b-e0e4c6239af8 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:59.784041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040e3108-156c-4089-b7b0-5d05a48e82c2 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:05:59.817580 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-f410ee2b-9cbb-45f3-939b-e0e4c6239af8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:05:59.817580 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:05:59.817580 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:05:59.817580 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:05:59.817580 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:05:59.817580 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:05:59.817580 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:05:59.817580 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:05:59.817580 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:05:59.817580 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:05:59.817580 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:05:59.817580 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:05:59.817580 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:05:59.817580 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:05:59.817580 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:05:59.818530 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:05:59.818530 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:05:59.818530 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:05:59.818530 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:05:59.818530 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:05:59.818530 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:05:59.818530 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:05:59.818530 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:05:59.818530 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:05:59.818530 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:05:59.818530 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:05:59.818530 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:05:59.818530 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:05:59.818530 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:05:59.818530 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:05:59.818530 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:05:59.818530 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:05:59.819431 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:05:59.819431 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:05:59.819431 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:05:59.819431 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:05:59.820625 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 281/545] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:05:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:05:59.827133 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f410ee2b-9cbb-45f3-939b-e0e4c6239af8 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:05:59.828739 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d45a4-427f-403d-a1ad-d419c11c8823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:05:59.829650 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d45a4-427f-403d-a1ad-d419c11c8823 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:59.830150 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d45a4-427f-403d-a1ad-d419c11c8823 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:59.830623 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d45a4-427f-403d-a1ad-d419c11c8823 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:59.831656 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0d45a4-427f-403d-a1ad-d419c11c8823 None admin] RBAC: Adding query filter params (role.id=acf4e26e0ceb41ddad002dd4edf6373c, scope.project.id=2a4699d1ec70456da5f8e0aa4b1439b7, user.id=59c7f07e58ba4db9a7d402d8f3da1e89) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:05:59.832190 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0d45a4-427f-403d-a1ad-d419c11c8823 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:05:59.833850 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0d45a4-427f-403d-a1ad-d419c11c8823 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:05:59.841036 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 265/546] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jun 21 18:05:59 2022] GET /identity/v3/role_assignments?user.id=59c7f07e58ba4db9a7d402d8f3da1e89&scope.project.id=2a4699d1ec70456da5f8e0aa4b1439b7&role.id=acf4e26e0ceb41ddad002dd4edf6373c => generated 259 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:05:59.887214 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9175969-121d-439e-a00f-de34c37bcf5f None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:59.887681 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9175969-121d-439e-a00f-de34c37bcf5f None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:59.888139 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9175969-121d-439e-a00f-de34c37bcf5f None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:59.890696 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 266/547] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:05:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:05:59.899275 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd53a87-6f6a-4300-ab1e-ae826e327436 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:05:59.899719 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd53a87-6f6a-4300-ab1e-ae826e327436 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:05:59.900262 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd53a87-6f6a-4300-ab1e-ae826e327436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:05:59.929509 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-ffd53a87-6f6a-4300-ab1e-ae826e327436 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:05:59.932005 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-ffd53a87-6f6a-4300-ab1e-ae826e327436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:06:00.056632 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 267/548] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:05:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:06:00.101612 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd53a87-6f6a-4300-ab1e-ae826e327436 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:00.158265 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f410ee2b-9cbb-45f3-939b-e0e4c6239af8 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:00.192610 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:00.208893 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:00.250550 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea44263-84a4-43b0-a5f6-bdad09663db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:00.251238 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea44263-84a4-43b0-a5f6-bdad09663db5 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:00.251524 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea44263-84a4-43b0-a5f6-bdad09663db5 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:00.251805 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea44263-84a4-43b0-a5f6-bdad09663db5 None admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:00.252569 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea44263-84a4-43b0-a5f6-bdad09663db5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:00.252882 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea44263-84a4-43b0-a5f6-bdad09663db5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:00.254888 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea44263-84a4-43b0-a5f6-bdad09663db5 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:00.275040 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 282/549] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:05:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 449 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:00.280920 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea44263-84a4-43b0-a5f6-bdad09663db5 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:00.283431 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd53a87-6f6a-4300-ab1e-ae826e327436 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:00.365204 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996b7c64-0ef8-4394-a7ea-a6a27cc0a2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:00.365945 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b7c64-0ef8-4394-a7ea-a6a27cc0a2a6 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:00.366232 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b7c64-0ef8-4394-a7ea-a6a27cc0a2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:00.366512 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b7c64-0ef8-4394-a7ea-a6a27cc0a2a6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:00.390105 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996b7c64-0ef8-4394-a7ea-a6a27cc0a2a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:06:00.401765 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea44263-84a4-43b0-a5f6-bdad09663db5 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:00.406325 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-996b7c64-0ef8-4394-a7ea-a6a27cc0a2a6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:06:00.406325 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:00.406325 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:00.406325 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:00.406325 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:00.406325 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:00.406325 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:00.406325 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:00.406325 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:00.406325 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:00.406325 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:00.406325 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:00.406325 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:06:00.406325 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:06:00.406325 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:06:00.406325 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:06:00.407286 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:06:00.407286 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:06:00.407286 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:06:00.407286 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:06:00.407286 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:00.407286 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:00.407286 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:00.407286 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:00.407286 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:00.407286 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:00.407286 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:00.407286 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:00.407286 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:00.407286 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:00.407286 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:00.407286 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:00.408125 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:00.408125 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:00.408125 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:00.408125 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:00.408125 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:06:00.408125 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:06:00.408125 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:06:00.408125 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:06:00.408125 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:06:00.408125 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:06:00.408125 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:06:00.408125 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:00.408125 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 268/550] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jun 21 18:06:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:00.424190 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996b7c64-0ef8-4394-a7ea-a6a27cc0a2a6 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:00.444858 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c9f783-632d-439c-83e9-7682e0a7272c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:00.445668 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c9f783-632d-439c-83e9-7682e0a7272c None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:00.445952 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c9f783-632d-439c-83e9-7682e0a7272c None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:00.447303 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c9f783-632d-439c-83e9-7682e0a7272c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:00.455641 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c9f783-632d-439c-83e9-7682e0a7272c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:00.457278 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c9f783-632d-439c-83e9-7682e0a7272c None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:00.476001 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-b7c9f783-632d-439c-83e9-7682e0a7272c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 21 18:06:00.476001 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:00.476001 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:00.476001 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:00.476001 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:00.476001 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:00.476001 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:00.476001 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:00.476001 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:00.476001 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:00.476001 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:00.476001 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:00.476001 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:06:00.476001 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:06:00.476001 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:06:00.476001 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:06:00.478087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:00.478087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:00.478087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:06:00.478087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:06:00.478087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:06:00.478087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:06:00.478087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:00.478087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:00.478087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:00.478087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:00.478087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:00.478087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:00.478087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:00.478087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:00.478087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:00.478087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:00.478087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:00.478087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:00.479927 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:00.479927 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:00.479927 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:06:00.479927 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:06:00.479927 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:06:00.479927 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:06:00.479927 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:06:00.479927 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:06:00.479927 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 21 18:06:00.479927 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:00.479927 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 283/551] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jun 21 18:06:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:06:00.485802 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c9f783-632d-439c-83e9-7682e0a7272c None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:00.526877 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:00.608879 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c9f783-632d-439c-83e9-7682e0a7272c None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:00.652952 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38841d15-74bc-42e6-a73e-c6696047921e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:00.653783 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38841d15-74bc-42e6-a73e-c6696047921e None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:00.654073 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38841d15-74bc-42e6-a73e-c6696047921e None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:00.654353 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38841d15-74bc-42e6-a73e-c6696047921e None admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:00.655205 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38841d15-74bc-42e6-a73e-c6696047921e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:00.655515 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38841d15-74bc-42e6-a73e-c6696047921e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:00.657147 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38841d15-74bc-42e6-a73e-c6696047921e None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:00.666583 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996b7c64-0ef8-4394-a7ea-a6a27cc0a2a6 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:00.683718 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:00.703934 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 284/552] 10.222.0.23 () {56 vars in 1135 bytes} [Tue Jun 21 18:06:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:06:00.709196 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38841d15-74bc-42e6-a73e-c6696047921e None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:00.747552 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa49d3d-976f-4e39-9000-38aea2448d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:00.757319 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa49d3d-976f-4e39-9000-38aea2448d00 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:00.757319 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa49d3d-976f-4e39-9000-38aea2448d00 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:00.757319 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa49d3d-976f-4e39-9000-38aea2448d00 None admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:00.757319 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa49d3d-976f-4e39-9000-38aea2448d00 None admin] RBAC: Adding query filter params (name=service) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:00.757319 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa49d3d-976f-4e39-9000-38aea2448d00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:00.757319 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa49d3d-976f-4e39-9000-38aea2448d00 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:00.771521 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 269/553] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jun 21 18:06:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:00.777240 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa49d3d-976f-4e39-9000-38aea2448d00 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:00.831355 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:00.841809 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38841d15-74bc-42e6-a73e-c6696047921e None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:00.879857 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c387ddf-6a80-4c6b-ad81-e2ed1b6d7bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:00.880562 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c387ddf-6a80-4c6b-ad81-e2ed1b6d7bf5 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:00.880952 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c387ddf-6a80-4c6b-ad81-e2ed1b6d7bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:00.881240 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c387ddf-6a80-4c6b-ad81-e2ed1b6d7bf5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:00.964135 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa49d3d-976f-4e39-9000-38aea2448d00 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:00.969143 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c387ddf-6a80-4c6b-ad81-e2ed1b6d7bf5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:00.970631 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c387ddf-6a80-4c6b-ad81-e2ed1b6d7bf5 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:01.001284 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08a02a2-5b1c-40bb-9612-9b6ee5e87e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:01.002193 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08a02a2-5b1c-40bb-9612-9b6ee5e87e18 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:01.002484 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08a02a2-5b1c-40bb-9612-9b6ee5e87e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:01.002761 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08a02a2-5b1c-40bb-9612-9b6ee5e87e18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:01.014460 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08a02a2-5b1c-40bb-9612-9b6ee5e87e18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:01.015936 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08a02a2-5b1c-40bb-9612-9b6ee5e87e18 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:01.034773 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-e08a02a2-5b1c-40bb-9612-9b6ee5e87e18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:01.034773 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:01.034773 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:01.034773 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:01.034773 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:01.034773 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:01.034773 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:01.034773 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:01.034773 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:01.034773 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:01.034773 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:01.034773 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:01.034773 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:06:01.034773 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:06:01.034773 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:06:01.035730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:06:01.035730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:01.035730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:01.035730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:01.035730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:01.035730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:01.035730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:01.035730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:01.035730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:01.035730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:01.035730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:01.035730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:01.035730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:01.035730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:01.035730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:01.035730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:01.035730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:01.036656 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:06:01.036656 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:06:01.036656 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:01.036656 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:01.036656 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 270/554] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:06:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:06:01.039505 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dddc6c3-9cf0-45fe-805e-300abf6c7f3d None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:01.039505 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dddc6c3-9cf0-45fe-805e-300abf6c7f3d None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:01.039505 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dddc6c3-9cf0-45fe-805e-300abf6c7f3d None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:01.039926 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 271/555] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:06:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:06:01.045420 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dddc6c3-9cf0-45fe-805e-300abf6c7f3d None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:01.074419 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:01.093344 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:01.109387 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 285/556] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:06:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 21 18:06:01.109387 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec55977f-5bfb-4ea1-9fd3-2961b27c52a6 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:01.109387 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec55977f-5bfb-4ea1-9fd3-2961b27c52a6 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:01.109387 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec55977f-5bfb-4ea1-9fd3-2961b27c52a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:01.129868 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-ec55977f-5bfb-4ea1-9fd3-2961b27c52a6 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:06:01.131452 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-ec55977f-5bfb-4ea1-9fd3-2961b27c52a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:06:01.196243 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dddc6c3-9cf0-45fe-805e-300abf6c7f3d None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:01.222805 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 286/557] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:06:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:06:01.223891 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec55977f-5bfb-4ea1-9fd3-2961b27c52a6 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:01.256012 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5788e61e-4e89-4e1a-b897-232908b93ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:01.257138 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788e61e-4e89-4e1a-b897-232908b93ea9 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:01.257575 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788e61e-4e89-4e1a-b897-232908b93ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:01.258359 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788e61e-4e89-4e1a-b897-232908b93ea9 None admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:01.259628 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5788e61e-4e89-4e1a-b897-232908b93ea9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:01.260166 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5788e61e-4e89-4e1a-b897-232908b93ea9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:01.262434 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5788e61e-4e89-4e1a-b897-232908b93ea9 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:01.277696 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 272/558] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:06:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:01.280326 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5788e61e-4e89-4e1a-b897-232908b93ea9 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:01.344941 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec55977f-5bfb-4ea1-9fd3-2961b27c52a6 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:01.372153 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:01.373085 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:01.385450 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d5d689-4a4f-420c-ba3a-740088c59ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:01.386341 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d5d689-4a4f-420c-ba3a-740088c59ea8 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:01.386697 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d5d689-4a4f-420c-ba3a-740088c59ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:01.387053 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d5d689-4a4f-420c-ba3a-740088c59ea8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:01.391364 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5788e61e-4e89-4e1a-b897-232908b93ea9 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:01.397232 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d5d689-4a4f-420c-ba3a-740088c59ea8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:01.398789 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d5d689-4a4f-420c-ba3a-740088c59ea8 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:01.418720 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-78d5d689-4a4f-420c-ba3a-740088c59ea8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:01.418720 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:01.418720 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:01.418720 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:01.418720 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:01.418720 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:01.418720 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:01.418720 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:01.418720 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:01.418720 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:01.418720 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:01.418720 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:01.418720 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:06:01.418720 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:06:01.418720 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:06:01.419680 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:06:01.419680 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:01.419680 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:01.419680 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:01.419680 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:01.419680 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:01.419680 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:01.419680 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:01.419680 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:01.419680 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:01.419680 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:01.419680 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:01.419680 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:01.419680 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:01.419680 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:01.419680 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:01.419680 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:01.420604 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:06:01.420604 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:06:01.420604 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:01.420604 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:01.420604 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 287/559] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:06:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:01.420896 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d5d689-4a4f-420c-ba3a-740088c59ea8 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:01.433714 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf28a7a-3b1f-4dbf-a9e2-73f4cce8c956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:01.434528 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf28a7a-3b1f-4dbf-a9e2-73f4cce8c956 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:01.434814 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf28a7a-3b1f-4dbf-a9e2-73f4cce8c956 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:01.435089 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf28a7a-3b1f-4dbf-a9e2-73f4cce8c956 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:01.446619 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf28a7a-3b1f-4dbf-a9e2-73f4cce8c956 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:06:01.454705 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-fbf28a7a-3b1f-4dbf-a9e2-73f4cce8c956 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:06:01.454705 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:01.454705 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:01.454705 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:01.454705 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:01.454705 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:01.454705 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:01.454705 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:01.454705 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:01.454705 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:01.454705 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:01.454705 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:01.454705 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:06:01.454705 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:06:01.454705 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:06:01.454705 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:06:01.455750 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:06:01.455750 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:06:01.455750 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:06:01.455750 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:06:01.455750 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:01.455750 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:01.455750 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:01.455750 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:01.455750 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:01.455750 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:01.455750 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:01.455750 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:01.455750 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:01.455750 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:01.455750 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:01.455750 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:01.457387 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:01.457387 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:01.457387 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:01.457387 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:01.457387 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:06:01.457387 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:06:01.457387 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:06:01.457387 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:06:01.457387 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:06:01.457387 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:06:01.457387 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:06:01.457387 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:01.457387 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 273/560] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jun 21 18:06:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:01.458128 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de79189-a11b-4591-b872-d83e2dc6df8d None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:01.458128 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de79189-a11b-4591-b872-d83e2dc6df8d None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:01.458250 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de79189-a11b-4591-b872-d83e2dc6df8d None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:01.464691 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 274/561] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:06:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:06:01.464691 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de79189-a11b-4591-b872-d83e2dc6df8d None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:01.542027 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d5d689-4a4f-420c-ba3a-740088c59ea8 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:01.578930 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450e1215-8a82-4ad0-b037-15e1cb86b4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:01.579141 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de79189-a11b-4591-b872-d83e2dc6df8d None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:01.579737 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e1215-8a82-4ad0-b037-15e1cb86b4f7 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:01.580024 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e1215-8a82-4ad0-b037-15e1cb86b4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:01.580304 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e1215-8a82-4ad0-b037-15e1cb86b4f7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:01.588893 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450e1215-8a82-4ad0-b037-15e1cb86b4f7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:01.590402 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450e1215-8a82-4ad0-b037-15e1cb86b4f7 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:01.606062 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-450e1215-8a82-4ad0-b037-15e1cb86b4f7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 21 18:06:01.606062 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:01.606062 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:01.606062 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:01.606062 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:01.606062 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:01.606062 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:01.606062 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:01.606062 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:01.606062 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:01.606062 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:01.606062 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:01.606062 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:06:01.606062 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:06:01.606062 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:06:01.606062 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:06:01.607087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:01.607087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:01.607087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:06:01.607087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:06:01.607087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:06:01.607087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:06:01.607087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:01.607087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:01.607087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:01.607087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:01.607087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:01.607087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:01.607087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:01.607087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:01.607087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:01.607087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:01.607087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:01.607087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:01.608146 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:01.608146 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:01.608146 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:06:01.608146 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:06:01.608146 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:06:01.608146 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:06:01.608146 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:06:01.608146 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:06:01.608146 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 21 18:06:01.608146 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:01.608146 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 288/562] 10.222.0.23 () {56 vars in 1093 bytes} [Tue Jun 21 18:06:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:01.609480 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450e1215-8a82-4ad0-b037-15e1cb86b4f7 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:01.625274 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b148576-a22f-48ae-a987-809394b941bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:01.626428 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b148576-a22f-48ae-a987-809394b941bc None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:01.626866 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b148576-a22f-48ae-a987-809394b941bc None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:01.627269 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b148576-a22f-48ae-a987-809394b941bc None admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:01.628250 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b148576-a22f-48ae-a987-809394b941bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:01.628595 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b148576-a22f-48ae-a987-809394b941bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:01.630382 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b148576-a22f-48ae-a987-809394b941bc None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:01.645518 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:01.652696 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 275/563] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:06:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:01.652696 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fd6173-b397-48e7-b055-88c024761c22 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:01.652696 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fd6173-b397-48e7-b055-88c024761c22 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:01.652696 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fd6173-b397-48e7-b055-88c024761c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:01.658064 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:01.674336 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-13fd6173-b397-48e7-b055-88c024761c22 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:06:01.676563 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-13fd6173-b397-48e7-b055-88c024761c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:06:01.716205 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450e1215-8a82-4ad0-b037-15e1cb86b4f7 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:01.750021 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 276/564] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:06:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:06:01.751251 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fd6173-b397-48e7-b055-88c024761c22 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:01.766142 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04db462-fb69-40a9-b26f-f0022c998ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:01.767314 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04db462-fb69-40a9-b26f-f0022c998ae6 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:01.767788 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04db462-fb69-40a9-b26f-f0022c998ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:01.768259 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04db462-fb69-40a9-b26f-f0022c998ae6 None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:01.770107 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04db462-fb69-40a9-b26f-f0022c998ae6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:01.770437 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04db462-fb69-40a9-b26f-f0022c998ae6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:01.772962 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04db462-fb69-40a9-b26f-f0022c998ae6 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:01.789543 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 289/565] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jun 21 18:06:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:01.791796 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04db462-fb69-40a9-b26f-f0022c998ae6 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:01.902144 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fd6173-b397-48e7-b055-88c024761c22 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:01.924776 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:01.931082 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04db462-fb69-40a9-b26f-f0022c998ae6 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:01.938901 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:01.946164 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559b1fa3-2e7b-4b2c-b172-9722107dfcd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:01.947913 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b1fa3-2e7b-4b2c-b172-9722107dfcd6 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:01.948267 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b1fa3-2e7b-4b2c-b172-9722107dfcd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:01.948635 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b1fa3-2e7b-4b2c-b172-9722107dfcd6 None admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:01.949823 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559b1fa3-2e7b-4b2c-b172-9722107dfcd6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:01.950142 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559b1fa3-2e7b-4b2c-b172-9722107dfcd6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:01.952218 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559b1fa3-2e7b-4b2c-b172-9722107dfcd6 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:01.970512 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8913f67-e904-4854-8c42-206b75d12985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:01.971457 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8913f67-e904-4854-8c42-206b75d12985 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:01.971764 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8913f67-e904-4854-8c42-206b75d12985 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:01.972076 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8913f67-e904-4854-8c42-206b75d12985 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:01.975623 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 277/566] 10.222.0.23 () {56 vars in 1133 bytes} [Tue Jun 21 18:06:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:01.976605 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559b1fa3-2e7b-4b2c-b172-9722107dfcd6 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:01.985006 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8913f67-e904-4854-8c42-206b75d12985 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:01.986688 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8913f67-e904-4854-8c42-206b75d12985 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:02.003723 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-b8913f67-e904-4854-8c42-206b75d12985 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:06:02.003723 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:02.003723 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:02.003723 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:02.003723 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:02.003723 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:02.003723 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:02.003723 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:02.003723 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:02.003723 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:02.003723 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:02.003723 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:02.003723 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:06:02.003723 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:06:02.003723 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:06:02.004920 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:06:02.004920 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:02.004920 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:02.004920 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:02.004920 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:02.004920 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:02.004920 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:02.004920 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:02.004920 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:02.004920 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:02.004920 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:02.004920 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:02.004920 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:02.004920 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:02.004920 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:02.004920 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:02.004920 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:02.006051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:06:02.006051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:06:02.006051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:06:02.006051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:06:02.006051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:06:02.006051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:06:02.006051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:06:02.006051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:02.006817 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 290/567] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jun 21 18:06:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:02.013057 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8913f67-e904-4854-8c42-206b75d12985 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:02.097617 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559b1fa3-2e7b-4b2c-b172-9722107dfcd6 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:02.127982 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8913f67-e904-4854-8c42-206b75d12985 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:02.137596 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c9d562-b94e-4d93-8980-309a315c05a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:02.138326 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9d562-b94e-4d93-8980-309a315c05a2 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:02.138609 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9d562-b94e-4d93-8980-309a315c05a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:02.138883 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9d562-b94e-4d93-8980-309a315c05a2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:02.151701 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c9d562-b94e-4d93-8980-309a315c05a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:06:02.160406 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-82c9d562-b94e-4d93-8980-309a315c05a2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:06:02.160406 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:02.160406 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:02.160406 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:02.160406 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:02.160406 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:02.160406 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:02.160406 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:02.160406 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:02.160406 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:02.160406 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:02.160406 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:02.160406 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:06:02.160406 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:06:02.160406 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:06:02.160406 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:06:02.161449 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:06:02.161449 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:06:02.161449 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:06:02.161449 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:06:02.161449 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:02.161449 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:02.161449 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:02.161449 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:02.161449 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:02.161449 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:02.161449 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:02.161449 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:02.161449 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:02.161449 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:02.161449 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:02.161449 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:02.162338 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:02.162338 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:02.162338 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:02.162338 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:02.162338 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:06:02.162338 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:06:02.162338 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:06:02.162338 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:06:02.162338 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:06:02.162338 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:06:02.162338 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:06:02.162338 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:02.162338 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 278/568] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jun 21 18:06:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:02.163458 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c9d562-b94e-4d93-8980-309a315c05a2 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:02.170913 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7163cc-7772-4e85-b642-47cf75f00352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:02.171736 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7163cc-7772-4e85-b642-47cf75f00352 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:02.172022 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7163cc-7772-4e85-b642-47cf75f00352 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:02.172301 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7163cc-7772-4e85-b642-47cf75f00352 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:02.184326 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7163cc-7772-4e85-b642-47cf75f00352 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:02.185905 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7163cc-7772-4e85-b642-47cf75f00352 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:02.205201 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-fb7163cc-7772-4e85-b642-47cf75f00352 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:02.205201 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:02.205201 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:02.205201 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:02.205201 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:02.205201 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:02.205201 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:02.205201 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:02.205201 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:02.205201 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:02.205201 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:02.205201 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:02.205201 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:06:02.205201 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:06:02.205201 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:06:02.206197 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:06:02.206197 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:02.206197 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:02.206197 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:02.206197 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:02.206197 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:02.206197 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:02.206197 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:02.206197 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:02.206197 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:02.206197 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:02.206197 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:02.206197 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:02.206197 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:02.206197 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:02.206197 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:02.206197 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:02.207116 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:06:02.207116 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:06:02.207116 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:02.207116 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:02.207116 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 291/569] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:06:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:02.207418 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7163cc-7772-4e85-b642-47cf75f00352 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:02.225819 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:02.233384 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:02.296834 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c9d562-b94e-4d93-8980-309a315c05a2 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:02.328324 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7163cc-7772-4e85-b642-47cf75f00352 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:02.337018 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46337d9f-dda2-4875-90d9-19f8a710a7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:02.338534 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46337d9f-dda2-4875-90d9-19f8a710a7ea None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:02.338998 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46337d9f-dda2-4875-90d9-19f8a710a7ea None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:02.339287 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46337d9f-dda2-4875-90d9-19f8a710a7ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:02.351333 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46337d9f-dda2-4875-90d9-19f8a710a7ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:02.353007 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46337d9f-dda2-4875-90d9-19f8a710a7ea None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:02.371939 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-46337d9f-dda2-4875-90d9-19f8a710a7ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:02.371939 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:02.371939 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:02.371939 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:02.371939 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:02.371939 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:02.371939 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:02.371939 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:02.371939 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:02.371939 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:02.371939 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:02.371939 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:02.371939 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:06:02.371939 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:06:02.371939 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:06:02.372966 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:06:02.372966 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:02.372966 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:02.372966 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:02.372966 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:02.372966 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:02.372966 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:02.372966 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:02.372966 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:02.372966 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:02.372966 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:02.372966 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:02.372966 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:02.372966 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:02.372966 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:02.372966 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:02.372966 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:02.373877 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:06:02.373877 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:06:02.373877 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:02.373877 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:02.373877 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 279/570] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:06:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:02.374269 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46337d9f-dda2-4875-90d9-19f8a710a7ea None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:02.386235 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8e073d-9501-4179-a9f8-9e69900a9a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:02.386940 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8e073d-9501-4179-a9f8-9e69900a9a28 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:02.387217 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8e073d-9501-4179-a9f8-9e69900a9a28 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:02.387534 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8e073d-9501-4179-a9f8-9e69900a9a28 None admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:02.388429 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8e073d-9501-4179-a9f8-9e69900a9a28 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:02.388695 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8e073d-9501-4179-a9f8-9e69900a9a28 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:02.390156 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8e073d-9501-4179-a9f8-9e69900a9a28 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:02.403576 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 292/571] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jun 21 18:06:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:02.404450 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8e073d-9501-4179-a9f8-9e69900a9a28 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:02.487702 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:02.501467 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46337d9f-dda2-4875-90d9-19f8a710a7ea None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:02.503062 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:02.530213 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8e073d-9501-4179-a9f8-9e69900a9a28 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:02.546511 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f49270-6942-4bfb-a409-7073084e688b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:02.547125 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f49270-6942-4bfb-a409-7073084e688b None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:02.547457 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f49270-6942-4bfb-a409-7073084e688b None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:02.547733 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f49270-6942-4bfb-a409-7073084e688b None admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:02.548529 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f49270-6942-4bfb-a409-7073084e688b None admin] RBAC: Adding query filter params (name=service) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:02.548835 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f49270-6942-4bfb-a409-7073084e688b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:02.550272 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f49270-6942-4bfb-a409-7073084e688b None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:02.559032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 280/572] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jun 21 18:06:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:02.559716 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f49270-6942-4bfb-a409-7073084e688b None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:02.570987 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31786934-2218-4a4f-9107-461c0361c397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:02.572627 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31786934-2218-4a4f-9107-461c0361c397 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:02.572916 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31786934-2218-4a4f-9107-461c0361c397 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:02.573196 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31786934-2218-4a4f-9107-461c0361c397 None admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:02.573940 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31786934-2218-4a4f-9107-461c0361c397 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:02.574247 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31786934-2218-4a4f-9107-461c0361c397 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:02.575707 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31786934-2218-4a4f-9107-461c0361c397 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:02.589503 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 293/573] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:06:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:02.590365 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31786934-2218-4a4f-9107-461c0361c397 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:02.679730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f49270-6942-4bfb-a409-7073084e688b None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:02.710983 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31786934-2218-4a4f-9107-461c0361c397 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:02.715143 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507b77b0-4b8f-42e4-a61c-c36e1154557d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:02.715817 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507b77b0-4b8f-42e4-a61c-c36e1154557d None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:02.716104 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507b77b0-4b8f-42e4-a61c-c36e1154557d None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:02.716400 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507b77b0-4b8f-42e4-a61c-c36e1154557d None admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:02.717147 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507b77b0-4b8f-42e4-a61c-c36e1154557d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:02.717454 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507b77b0-4b8f-42e4-a61c-c36e1154557d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:02.718848 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507b77b0-4b8f-42e4-a61c-c36e1154557d None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:02.731573 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 281/574] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:06:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:02.732966 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c7f73-84c2-45e8-8377-86ed20149f4c None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:02.733260 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c7f73-84c2-45e8-8377-86ed20149f4c None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:02.733554 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c7f73-84c2-45e8-8377-86ed20149f4c None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:02.735030 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 282/575] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:06:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:06:02.735629 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0c7f73-84c2-45e8-8377-86ed20149f4c None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:02.742414 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:02.751025 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56a3d5-9292-4578-b132-e03e2fe6b15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:02.751694 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56a3d5-9292-4578-b132-e03e2fe6b15b None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:02.758341 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56a3d5-9292-4578-b132-e03e2fe6b15b None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:02.758341 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56a3d5-9292-4578-b132-e03e2fe6b15b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:02.758341 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e56a3d5-9292-4578-b132-e03e2fe6b15b None admin] RBAC: Adding query filter params (role.id=acf4e26e0ceb41ddad002dd4edf6373c, scope.project.id=2a4699d1ec70456da5f8e0aa4b1439b7, user.id=0e0d25565f1543d9a7577288fc09119f) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:02.758341 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e56a3d5-9292-4578-b132-e03e2fe6b15b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:02.758341 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e56a3d5-9292-4578-b132-e03e2fe6b15b None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:02.764648 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:02.769476 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 294/576] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jun 21 18:06:02 2022] GET /identity/v3/role_assignments?user.id=0e0d25565f1543d9a7577288fc09119f&scope.project.id=2a4699d1ec70456da5f8e0aa4b1439b7&role.id=acf4e26e0ceb41ddad002dd4edf6373c => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:02.770325 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56a3d5-9292-4578-b132-e03e2fe6b15b None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:02.888582 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0c7f73-84c2-45e8-8377-86ed20149f4c None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:02.918547 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56a3d5-9292-4578-b132-e03e2fe6b15b None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:02.936207 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b261ac2b-b5e0-41a9-a309-a236832ae166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:02.936980 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b261ac2b-b5e0-41a9-a309-a236832ae166 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:02.937887 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b261ac2b-b5e0-41a9-a309-a236832ae166 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:02.937887 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b261ac2b-b5e0-41a9-a309-a236832ae166 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:02.950359 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b261ac2b-b5e0-41a9-a309-a236832ae166 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:02.952147 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b261ac2b-b5e0-41a9-a309-a236832ae166 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:02.964653 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d717db-e302-4d45-bd0f-9aa12876ef42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:02.967229 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d717db-e302-4d45-bd0f-9aa12876ef42 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:02.967229 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d717db-e302-4d45-bd0f-9aa12876ef42 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:02.967229 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d717db-e302-4d45-bd0f-9aa12876ef42 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:02.973613 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-b261ac2b-b5e0-41a9-a309-a236832ae166 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:02.973613 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:02.973613 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:02.973613 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:02.973613 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:02.973613 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:02.973613 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:02.973613 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:02.973613 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:02.973613 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:02.973613 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:02.973613 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:02.973613 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:06:02.973613 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:06:02.973613 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:06:02.974591 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:06:02.974591 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:02.974591 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:02.974591 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:02.974591 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:02.974591 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:02.974591 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:02.974591 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:02.974591 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:02.974591 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:02.974591 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:02.974591 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:02.974591 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:02.974591 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:02.974591 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:02.974591 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:02.974591 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:02.975562 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:06:02.975562 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:06:02.975562 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:02.975562 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:02.975562 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 283/577] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:06:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:02.975562 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b261ac2b-b5e0-41a9-a309-a236832ae166 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:02.983188 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d717db-e302-4d45-bd0f-9aa12876ef42 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:02.984693 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d717db-e302-4d45-bd0f-9aa12876ef42 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:03.001417 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-c9d717db-e302-4d45-bd0f-9aa12876ef42 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 21 18:06:03.001417 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:03.001417 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:03.001417 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:03.001417 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:03.001417 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:03.001417 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:03.001417 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:03.001417 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:03.001417 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:03.001417 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:03.001417 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:03.001417 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:06:03.001417 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:06:03.001417 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:06:03.001417 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:06:03.002468 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:03.002468 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:03.002468 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:06:03.002468 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:06:03.002468 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:06:03.002468 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:06:03.002468 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:03.002468 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:03.002468 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:03.002468 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:03.002468 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:03.002468 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:03.002468 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:03.002468 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:03.002468 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:03.002468 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:03.002468 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:03.002468 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:03.003489 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:03.003489 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:03.003489 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:06:03.003489 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:06:03.003489 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:06:03.003489 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:06:03.003489 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:06:03.003489 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:06:03.003489 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 21 18:06:03.003489 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:03.003489 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 295/578] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:06:02 2022] GET /identity/v3/users/cinder => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:03.004115 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc4acda-f7a7-4bc7-b216-681b7ee6efd7 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:03.004115 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc4acda-f7a7-4bc7-b216-681b7ee6efd7 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:03.004798 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc4acda-f7a7-4bc7-b216-681b7ee6efd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:03.022559 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-dcc4acda-f7a7-4bc7-b216-681b7ee6efd7 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:06:03.023757 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-dcc4acda-f7a7-4bc7-b216-681b7ee6efd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:06:03.074068 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:03.074648 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:03.104264 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 296/579] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:06:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:06:03.104264 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc4acda-f7a7-4bc7-b216-681b7ee6efd7 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:03.113952 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b261ac2b-b5e0-41a9-a309-a236832ae166 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:03.150533 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831d2928-673c-437c-b325-ec186c15d3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:03.151347 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d2928-673c-437c-b325-ec186c15d3f8 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:03.151621 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d2928-673c-437c-b325-ec186c15d3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:03.151900 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d2928-673c-437c-b325-ec186c15d3f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:03.160033 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831d2928-673c-437c-b325-ec186c15d3f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:03.161553 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831d2928-673c-437c-b325-ec186c15d3f8 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:03.183061 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-831d2928-673c-437c-b325-ec186c15d3f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:06:03.183061 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:03.183061 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:03.183061 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:03.183061 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:03.183061 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:03.183061 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:03.183061 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:03.183061 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:03.183061 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:03.183061 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:03.183061 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:03.183061 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:06:03.183061 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:06:03.183061 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:06:03.184923 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:06:03.184923 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:03.184923 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:03.184923 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:03.184923 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:03.184923 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:03.184923 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:03.184923 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:03.184923 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:03.184923 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:03.184923 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:03.184923 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:03.184923 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:03.184923 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:03.184923 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:03.184923 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:03.184923 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:03.185871 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:06:03.185871 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:06:03.185871 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:06:03.185871 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:06:03.185871 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:06:03.185871 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:06:03.185871 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:06:03.185871 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:03.185871 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 284/580] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jun 21 18:06:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:03.190392 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831d2928-673c-437c-b325-ec186c15d3f8 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:03.242970 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc4acda-f7a7-4bc7-b216-681b7ee6efd7 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:03.283544 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9fdd7b-28b8-4066-ae05-526d93065b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:03.284218 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9fdd7b-28b8-4066-ae05-526d93065b63 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:03.284504 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9fdd7b-28b8-4066-ae05-526d93065b63 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:03.284781 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9fdd7b-28b8-4066-ae05-526d93065b63 None admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:03.285507 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9fdd7b-28b8-4066-ae05-526d93065b63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:03.285810 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9fdd7b-28b8-4066-ae05-526d93065b63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:03.287175 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9fdd7b-28b8-4066-ae05-526d93065b63 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:03.299521 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 297/581] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:06:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:03.300340 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9fdd7b-28b8-4066-ae05-526d93065b63 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:03.307013 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831d2928-673c-437c-b325-ec186c15d3f8 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:03.341961 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b33c639-c972-440d-b881-4bf17b5648b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:03.342639 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b33c639-c972-440d-b881-4bf17b5648b0 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:03.342925 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b33c639-c972-440d-b881-4bf17b5648b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:03.343196 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b33c639-c972-440d-b881-4bf17b5648b0 None admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:03.344068 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b33c639-c972-440d-b881-4bf17b5648b0 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:03.344364 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b33c639-c972-440d-b881-4bf17b5648b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:03.345809 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b33c639-c972-440d-b881-4bf17b5648b0 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:03.346675 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:03.349389 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:03.365859 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 285/582] 10.222.0.23 () {56 vars in 1137 bytes} [Tue Jun 21 18:06:03 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:03.366991 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b33c639-c972-440d-b881-4bf17b5648b0 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:03.417827 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9fdd7b-28b8-4066-ae05-526d93065b63 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:03.453057 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85264ba8-0871-4af0-a492-3d530178b01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:03.453854 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85264ba8-0871-4af0-a492-3d530178b01b None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:03.454055 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85264ba8-0871-4af0-a492-3d530178b01b None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:03.454299 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85264ba8-0871-4af0-a492-3d530178b01b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:03.466897 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85264ba8-0871-4af0-a492-3d530178b01b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:06:03.476046 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-85264ba8-0871-4af0-a492-3d530178b01b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:06:03.476046 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:03.476046 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:03.476046 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:03.476046 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:03.476046 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:03.476046 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:03.476046 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:03.476046 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:03.476046 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:03.476046 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:03.476046 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:03.476046 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:06:03.476046 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:06:03.476046 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:06:03.476046 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:06:03.477124 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:06:03.477124 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:06:03.477124 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:06:03.477124 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:06:03.477124 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:03.477124 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:03.477124 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:03.477124 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:03.477124 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:03.477124 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:03.477124 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:03.477124 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:03.477124 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:03.477124 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:03.477124 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:03.477124 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:03.478861 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:03.478861 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:03.478861 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:03.478861 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:03.478861 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:06:03.478861 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:06:03.478861 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:06:03.478861 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:06:03.478861 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:06:03.478861 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:06:03.478861 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:06:03.478861 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:03.478861 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 298/583] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jun 21 18:06:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:03.478861 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85264ba8-0871-4af0-a492-3d530178b01b None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:03.493129 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b33c639-c972-440d-b881-4bf17b5648b0 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:03.542462 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f6ea61-65fa-4d9a-9640-916a848c2935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:03.543423 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f6ea61-65fa-4d9a-9640-916a848c2935 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:03.543833 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f6ea61-65fa-4d9a-9640-916a848c2935 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:03.544617 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f6ea61-65fa-4d9a-9640-916a848c2935 None admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:03.545289 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f6ea61-65fa-4d9a-9640-916a848c2935 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:03.545637 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f6ea61-65fa-4d9a-9640-916a848c2935 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:03.547098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f6ea61-65fa-4d9a-9640-916a848c2935 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:03.563979 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 286/584] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jun 21 18:06:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:03.563979 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f6ea61-65fa-4d9a-9640-916a848c2935 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:03.616358 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85264ba8-0871-4af0-a492-3d530178b01b None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:03.617928 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:03.653924 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:03.665444 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99719cf6-863c-4512-826e-2967c5e5f2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:03.666318 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99719cf6-863c-4512-826e-2967c5e5f2ef None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:03.666602 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99719cf6-863c-4512-826e-2967c5e5f2ef None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:03.666891 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99719cf6-863c-4512-826e-2967c5e5f2ef None admin] PATH_INFO: `/v3/users/glance` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:03.674886 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99719cf6-863c-4512-826e-2967c5e5f2ef None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:03.676337 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99719cf6-863c-4512-826e-2967c5e5f2ef None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:03.690886 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-99719cf6-863c-4512-826e-2967c5e5f2ef None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 21 18:06:03.690886 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:03.690886 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:03.690886 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:03.690886 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:03.690886 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:03.690886 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:03.690886 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:03.690886 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:03.690886 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:03.690886 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:03.690886 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:03.690886 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:06:03.690886 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:06:03.690886 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:06:03.690886 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:06:03.692944 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:03.692944 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:03.692944 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:06:03.692944 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:06:03.692944 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:06:03.692944 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:06:03.692944 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:03.692944 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:03.692944 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:03.692944 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:03.692944 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:03.692944 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:03.692944 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:03.692944 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:03.692944 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:03.692944 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:03.692944 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:03.692944 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:03.693967 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:03.693967 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:03.693967 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:06:03.693967 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:06:03.693967 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:06:03.693967 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:06:03.693967 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:06:03.693967 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:06:03.693967 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 21 18:06:03.693967 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:03.693967 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 299/585] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:06:03 2022] GET /identity/v3/users/glance => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:03.693967 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99719cf6-863c-4512-826e-2967c5e5f2ef None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:03.711522 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f6ea61-65fa-4d9a-9640-916a848c2935 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:03.748785 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db16fd-eb4b-4b37-8bdc-d38974af7292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:03.749670 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db16fd-eb4b-4b37-8bdc-d38974af7292 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:03.750021 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db16fd-eb4b-4b37-8bdc-d38974af7292 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:03.750361 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db16fd-eb4b-4b37-8bdc-d38974af7292 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:03.762912 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6db16fd-eb4b-4b37-8bdc-d38974af7292 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:03.764447 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6db16fd-eb4b-4b37-8bdc-d38974af7292 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:03.782984 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-c6db16fd-eb4b-4b37-8bdc-d38974af7292 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:03.782984 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:03.782984 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:03.782984 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:03.782984 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:03.782984 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:03.782984 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:03.782984 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:03.782984 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:03.782984 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:03.782984 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:03.782984 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:03.782984 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:06:03.782984 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:06:03.782984 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:06:03.783959 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:06:03.783959 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:03.783959 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:03.783959 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:03.783959 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:03.783959 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:03.783959 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:03.783959 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:03.783959 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:03.783959 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:03.783959 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:03.783959 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:03.783959 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:03.783959 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:03.783959 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:03.783959 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:03.783959 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:03.785518 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:06:03.785518 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:06:03.785518 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:03.785518 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:03.785518 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 287/586] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:06:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:03.785518 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db16fd-eb4b-4b37-8bdc-d38974af7292 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:03.809603 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99719cf6-863c-4512-826e-2967c5e5f2ef None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:03.847649 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e433fcbc-e214-42be-a429-b323fdcf3648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:03.848942 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e433fcbc-e214-42be-a429-b323fdcf3648 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:03.849412 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e433fcbc-e214-42be-a429-b323fdcf3648 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:03.849896 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e433fcbc-e214-42be-a429-b323fdcf3648 None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:03.850684 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e433fcbc-e214-42be-a429-b323fdcf3648 None admin] RBAC: Adding query filter params (name=service) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:03.850999 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e433fcbc-e214-42be-a429-b323fdcf3648 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:03.852427 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e433fcbc-e214-42be-a429-b323fdcf3648 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:03.862049 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 300/587] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jun 21 18:06:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:03.862899 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e433fcbc-e214-42be-a429-b323fdcf3648 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:03.907840 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:03.920679 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db16fd-eb4b-4b37-8bdc-d38974af7292 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:03.928496 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:03.957807 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ef93f-1098-4959-a03b-0f789463c6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:03.958989 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ef93f-1098-4959-a03b-0f789463c6a4 None admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:03.958989 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ef93f-1098-4959-a03b-0f789463c6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:03.959252 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ef93f-1098-4959-a03b-0f789463c6a4 None admin] PATH_INFO: `/v3/projects/2a4699d1ec70456da5f8e0aa4b1439b7/users/904255dd3c1f406eb10f6fc65f9cd098/roles/acf4e26e0ceb41ddad002dd4edf6373c` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:03.977249 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10ef93f-1098-4959-a03b-0f789463c6a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a4699d1ec70456da5f8e0aa4b1439b7, user_id=904255dd3c1f406eb10f6fc65f9cd098, role_id=acf4e26e0ceb41ddad002dd4edf6373c)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:03.978880 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10ef93f-1098-4959-a03b-0f789463c6a4 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:04.011746 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e433fcbc-e214-42be-a429-b323fdcf3648 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:04.057376 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84edb972-7116-43fa-b2c8-c96417f5487b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:04.058078 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84edb972-7116-43fa-b2c8-c96417f5487b None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:04.058355 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84edb972-7116-43fa-b2c8-c96417f5487b None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:04.058625 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84edb972-7116-43fa-b2c8-c96417f5487b None admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:04.059482 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84edb972-7116-43fa-b2c8-c96417f5487b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:04.059794 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84edb972-7116-43fa-b2c8-c96417f5487b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:04.072790 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 288/588] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jun 21 18:06:03 2022] PUT /identity/v3/projects/2a4699d1ec70456da5f8e0aa4b1439b7/users/904255dd3c1f406eb10f6fc65f9cd098/roles/acf4e26e0ceb41ddad002dd4edf6373c => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:06:04.073002 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84edb972-7116-43fa-b2c8-c96417f5487b None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:04.073291 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ef93f-1098-4959-a03b-0f789463c6a4 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:04.090747 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 301/589] 10.222.0.23 () {56 vars in 1137 bytes} [Tue Jun 21 18:06:03 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:04.091786 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84edb972-7116-43fa-b2c8-c96417f5487b None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:04.211156 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:04.232545 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ef93f-1098-4959-a03b-0f789463c6a4 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:04.238505 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84edb972-7116-43fa-b2c8-c96417f5487b None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:04.281215 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131347f9-c1a5-46c2-a1f6-ed988ec15c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:04.281926 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131347f9-c1a5-46c2-a1f6-ed988ec15c62 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:04.282187 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131347f9-c1a5-46c2-a1f6-ed988ec15c62 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:04.282991 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131347f9-c1a5-46c2-a1f6-ed988ec15c62 None admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:04.283196 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131347f9-c1a5-46c2-a1f6-ed988ec15c62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:04.283515 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131347f9-c1a5-46c2-a1f6-ed988ec15c62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:04.284957 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131347f9-c1a5-46c2-a1f6-ed988ec15c62 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:04.290543 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11401ef-d2be-4323-b4ef-c730f7d55519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:04.291476 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11401ef-d2be-4323-b4ef-c730f7d55519 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:04.291831 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11401ef-d2be-4323-b4ef-c730f7d55519 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:04.292177 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11401ef-d2be-4323-b4ef-c730f7d55519 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:04.298978 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 289/590] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:06:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:04.299742 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131347f9-c1a5-46c2-a1f6-ed988ec15c62 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:04.309410 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11401ef-d2be-4323-b4ef-c730f7d55519 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:04.311939 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11401ef-d2be-4323-b4ef-c730f7d55519 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:04.320773 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:04.341028 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-a11401ef-d2be-4323-b4ef-c730f7d55519 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:04.341028 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:04.341028 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:04.341028 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:04.341028 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:04.341028 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:04.341028 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:04.341028 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:04.341028 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:04.341028 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:04.341028 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:04.341028 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:04.341028 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:06:04.341028 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:06:04.341028 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:06:04.343906 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:06:04.343906 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:04.343906 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:04.343906 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:04.343906 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:04.343906 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:04.343906 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:04.343906 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:04.343906 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:04.343906 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:04.343906 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:04.343906 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:04.343906 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:04.343906 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:04.343906 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:04.343906 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:04.343906 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:04.344881 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:06:04.344881 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:06:04.344881 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:04.344881 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:04.344881 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 302/591] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:06:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:04.346630 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11401ef-d2be-4323-b4ef-c730f7d55519 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:04.417659 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131347f9-c1a5-46c2-a1f6-ed988ec15c62 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:04.459143 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c44f36-ced9-4cfc-b5ab-789d6f5ec077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:04.460054 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c44f36-ced9-4cfc-b5ab-789d6f5ec077 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:04.460441 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c44f36-ced9-4cfc-b5ab-789d6f5ec077 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:04.460798 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c44f36-ced9-4cfc-b5ab-789d6f5ec077 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:04.472438 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c44f36-ced9-4cfc-b5ab-789d6f5ec077 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:04.474252 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c44f36-ced9-4cfc-b5ab-789d6f5ec077 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:04.492224 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11401ef-d2be-4323-b4ef-c730f7d55519 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:04.495114 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-04c44f36-ced9-4cfc-b5ab-789d6f5ec077 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:04.495114 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:04.495114 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:04.495114 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:04.495114 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:04.495114 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:04.495114 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:04.495114 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:04.495114 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:04.495114 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:04.495114 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:04.495114 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:04.495114 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:06:04.495114 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:06:04.495114 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:06:04.496257 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:06:04.496257 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:04.496257 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:04.496257 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:04.496257 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:04.496257 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:04.496257 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:04.496257 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:04.496257 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:04.496257 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:04.496257 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:04.496257 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:04.496257 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:04.496257 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:04.496257 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:04.496257 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:04.496257 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:04.497303 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:06:04.497303 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:06:04.497303 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:04.497303 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:04.500427 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 290/592] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:06:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:04.500427 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c44f36-ced9-4cfc-b5ab-789d6f5ec077 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:04.534451 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bf870c-3e8e-478d-aed9-56cbfce19a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:04.535348 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf870c-3e8e-478d-aed9-56cbfce19a66 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:04.535638 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf870c-3e8e-478d-aed9-56cbfce19a66 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:04.535917 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf870c-3e8e-478d-aed9-56cbfce19a66 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:04.538435 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:04.546588 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bf870c-3e8e-478d-aed9-56cbfce19a66 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:04.546790 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bf870c-3e8e-478d-aed9-56cbfce19a66 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:04.566691 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-f0bf870c-3e8e-478d-aed9-56cbfce19a66 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:06:04.566691 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:04.566691 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:04.566691 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:04.566691 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:04.566691 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:04.566691 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:04.566691 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:04.566691 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:04.566691 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:04.566691 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:04.566691 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:04.566691 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:06:04.566691 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:06:04.566691 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:06:04.567874 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:06:04.567874 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:04.567874 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:04.567874 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:04.567874 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:04.567874 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:04.567874 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:04.567874 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:04.567874 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:04.567874 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:04.567874 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:04.567874 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:04.567874 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:04.567874 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:04.567874 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:04.567874 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:04.567874 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:04.568909 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:06:04.568909 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:06:04.568909 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:06:04.568909 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:06:04.568909 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:06:04.568909 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:06:04.568909 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:06:04.568909 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:04.572563 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 303/593] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jun 21 18:06:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:04.573623 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bf870c-3e8e-478d-aed9-56cbfce19a66 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:04.601346 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:04.664630 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c44f36-ced9-4cfc-b5ab-789d6f5ec077 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:04.708970 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10137307-874d-4aa7-9f8e-d5abf031b72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:04.709650 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10137307-874d-4aa7-9f8e-d5abf031b72c None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:04.709940 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10137307-874d-4aa7-9f8e-d5abf031b72c None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:04.710212 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10137307-874d-4aa7-9f8e-d5abf031b72c None admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:04.710944 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10137307-874d-4aa7-9f8e-d5abf031b72c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:04.711273 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10137307-874d-4aa7-9f8e-d5abf031b72c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:04.713962 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10137307-874d-4aa7-9f8e-d5abf031b72c None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:04.725835 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bf870c-3e8e-478d-aed9-56cbfce19a66 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:04.728525 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 291/594] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:06:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:04.729631 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10137307-874d-4aa7-9f8e-d5abf031b72c None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:04.771956 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d277651-4b3c-49f8-a344-0bb3d3c184ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:04.772677 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d277651-4b3c-49f8-a344-0bb3d3c184ad None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:04.773038 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d277651-4b3c-49f8-a344-0bb3d3c184ad None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:04.773404 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d277651-4b3c-49f8-a344-0bb3d3c184ad None admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:04.774217 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d277651-4b3c-49f8-a344-0bb3d3c184ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:04.774598 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d277651-4b3c-49f8-a344-0bb3d3c184ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:04.776738 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d277651-4b3c-49f8-a344-0bb3d3c184ad None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:04.791177 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 304/595] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:06:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:04.792560 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d277651-4b3c-49f8-a344-0bb3d3c184ad None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:04.852057 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:04.859980 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10137307-874d-4aa7-9f8e-d5abf031b72c None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:04.891475 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:04.900705 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e957fb0-e225-4519-8067-879982dd74ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:04.904048 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e957fb0-e225-4519-8067-879982dd74ac None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:04.904048 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e957fb0-e225-4519-8067-879982dd74ac None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:04.904048 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e957fb0-e225-4519-8067-879982dd74ac None admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:04.904048 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e957fb0-e225-4519-8067-879982dd74ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:04.904048 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e957fb0-e225-4519-8067-879982dd74ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:04.904529 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e957fb0-e225-4519-8067-879982dd74ac None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:04.906760 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d277651-4b3c-49f8-a344-0bb3d3c184ad None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:04.916986 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 292/596] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jun 21 18:06:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:04.919068 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e957fb0-e225-4519-8067-879982dd74ac None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:04.944941 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a319e830-43cf-4450-9375-1e8b866ee613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:04.946193 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a319e830-43cf-4450-9375-1e8b866ee613 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:04.946485 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a319e830-43cf-4450-9375-1e8b866ee613 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:04.946921 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a319e830-43cf-4450-9375-1e8b866ee613 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:04.954567 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a319e830-43cf-4450-9375-1e8b866ee613 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:04.956018 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a319e830-43cf-4450-9375-1e8b866ee613 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:04.971327 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-a319e830-43cf-4450-9375-1e8b866ee613 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 21 18:06:04.971327 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:04.971327 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:04.971327 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:04.971327 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:04.971327 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:04.971327 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:04.971327 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:04.971327 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:04.971327 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:04.971327 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:04.971327 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:04.971327 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:06:04.971327 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:06:04.971327 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:06:04.971327 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:06:04.972538 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:04.972538 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:04.972538 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:06:04.972538 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:06:04.972538 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:06:04.972538 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:06:04.972538 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:04.972538 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:04.972538 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:04.972538 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:04.972538 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:04.972538 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:04.972538 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:04.972538 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:04.972538 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:04.972538 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:04.972538 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:04.972538 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:04.979410 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:04.979410 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:04.979410 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:06:04.979410 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:06:04.979410 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:06:04.979410 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:06:04.979410 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:06:04.979410 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:06:04.979410 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 21 18:06:04.979410 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:04.979410 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 305/597] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jun 21 18:06:04 2022] GET /identity/v3/users/neutron => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:04.979410 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d2d8c1-6024-4a73-a9e8-05eec3a3032a None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:04.979410 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d2d8c1-6024-4a73-a9e8-05eec3a3032a None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:04.980144 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d2d8c1-6024-4a73-a9e8-05eec3a3032a None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:04.980144 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 306/598] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:06:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:06:04.980144 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d2d8c1-6024-4a73-a9e8-05eec3a3032a None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:05.032927 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e957fb0-e225-4519-8067-879982dd74ac None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:05.069397 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50790fc-3446-4300-98e4-290b4bab5950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:05.070291 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50790fc-3446-4300-98e4-290b4bab5950 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:05.070577 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50790fc-3446-4300-98e4-290b4bab5950 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:05.070846 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50790fc-3446-4300-98e4-290b4bab5950 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:05.078592 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50790fc-3446-4300-98e4-290b4bab5950 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:05.080033 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50790fc-3446-4300-98e4-290b4bab5950 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:05.094916 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d2d8c1-6024-4a73-a9e8-05eec3a3032a None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:05.095124 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-e50790fc-3446-4300-98e4-290b4bab5950 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:06:05.095124 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:05.095124 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:05.095124 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:05.095124 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:05.095124 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:05.095124 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:05.095124 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:05.095124 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:05.095124 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:05.095124 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:05.095124 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:05.095124 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:06:05.095124 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:06:05.095124 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:06:05.096071 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:06:05.096071 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:05.096071 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:05.096071 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:05.096071 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:05.096071 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:05.096071 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:05.096071 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:05.096071 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:05.096071 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:05.096071 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:05.096071 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:05.096071 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:05.096071 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:05.096071 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:05.096071 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:05.096071 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:05.097448 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:06:05.097448 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:06:05.097448 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:06:05.097448 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:06:05.097448 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:06:05.097448 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:06:05.097448 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:06:05.097448 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:05.101231 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 293/599] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jun 21 18:06:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:05.101902 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50790fc-3446-4300-98e4-290b4bab5950 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:05.113106 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:05.135281 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2f1285-79dc-4766-bf81-b3f2c8907473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:05.136182 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f1285-79dc-4766-bf81-b3f2c8907473 None admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:05.136502 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f1285-79dc-4766-bf81-b3f2c8907473 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:05.136793 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f1285-79dc-4766-bf81-b3f2c8907473 None admin] PATH_INFO: `/v3/projects/2a4699d1ec70456da5f8e0aa4b1439b7/users/77c6e43555944902a22ba2fe17a8e99f/roles/acf4e26e0ceb41ddad002dd4edf6373c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:05.155577 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2f1285-79dc-4766-bf81-b3f2c8907473 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a4699d1ec70456da5f8e0aa4b1439b7, user_id=77c6e43555944902a22ba2fe17a8e99f, role_id=acf4e26e0ceb41ddad002dd4edf6373c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:05.156446 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:05.157064 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2f1285-79dc-4766-bf81-b3f2c8907473 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:05.207559 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50790fc-3446-4300-98e4-290b4bab5950 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:05.220866 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 307/600] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jun 21 18:06:04 2022] PUT /identity/v3/projects/2a4699d1ec70456da5f8e0aa4b1439b7/users/77c6e43555944902a22ba2fe17a8e99f/roles/acf4e26e0ceb41ddad002dd4edf6373c => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:06:05.222471 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c32bb-ec70-4e2b-beab-ab4d8d08878e None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:05.222921 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c32bb-ec70-4e2b-beab-ab4d8d08878e None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:05.223609 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c32bb-ec70-4e2b-beab-ab4d8d08878e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:05.243414 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-eb1c32bb-ec70-4e2b-beab-ab4d8d08878e None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:06:05.245089 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-eb1c32bb-ec70-4e2b-beab-ab4d8d08878e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:06:05.256326 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88acc191-bde6-4c57-a080-4e7b5b08eb6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:05.257076 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88acc191-bde6-4c57-a080-4e7b5b08eb6d None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:05.257358 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88acc191-bde6-4c57-a080-4e7b5b08eb6d None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:05.257633 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88acc191-bde6-4c57-a080-4e7b5b08eb6d None admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:05.258552 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88acc191-bde6-4c57-a080-4e7b5b08eb6d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:05.259084 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88acc191-bde6-4c57-a080-4e7b5b08eb6d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:05.261407 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88acc191-bde6-4c57-a080-4e7b5b08eb6d None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:05.288812 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 294/601] 10.222.0.23 () {56 vars in 1139 bytes} [Tue Jun 21 18:06:05 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:05.289855 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88acc191-bde6-4c57-a080-4e7b5b08eb6d None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:05.344573 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 308/602] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:06:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:06:05.345953 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c32bb-ec70-4e2b-beab-ab4d8d08878e None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:05.447621 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88acc191-bde6-4c57-a080-4e7b5b08eb6d None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:05.458394 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:05.464957 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:05.474838 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c32bb-ec70-4e2b-beab-ab4d8d08878e None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:05.489758 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028f662c-3ddd-4353-8ab8-de578bd045a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:05.490675 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f662c-3ddd-4353-8ab8-de578bd045a9 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:05.491126 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f662c-3ddd-4353-8ab8-de578bd045a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:05.491558 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f662c-3ddd-4353-8ab8-de578bd045a9 None admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:05.492514 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028f662c-3ddd-4353-8ab8-de578bd045a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:05.493377 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028f662c-3ddd-4353-8ab8-de578bd045a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:05.495336 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028f662c-3ddd-4353-8ab8-de578bd045a9 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:05.507966 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 295/603] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jun 21 18:06:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:05.512420 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028f662c-3ddd-4353-8ab8-de578bd045a9 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:05.517443 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194ae4a3-480d-402a-9046-6942596ca479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:05.517443 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ae4a3-480d-402a-9046-6942596ca479 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:05.517443 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ae4a3-480d-402a-9046-6942596ca479 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:05.517744 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ae4a3-480d-402a-9046-6942596ca479 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:05.532281 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194ae4a3-480d-402a-9046-6942596ca479 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:05.534795 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194ae4a3-480d-402a-9046-6942596ca479 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:05.561122 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-194ae4a3-480d-402a-9046-6942596ca479 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:05.561122 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:05.561122 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:05.561122 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:05.561122 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:05.561122 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:05.561122 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:05.561122 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:05.561122 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:05.561122 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:05.561122 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:05.561122 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:05.561122 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:06:05.561122 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:06:05.561122 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:06:05.562160 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:06:05.562160 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:05.562160 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:05.562160 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:05.562160 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:05.562160 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:05.562160 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:05.562160 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:05.562160 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:05.562160 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:05.562160 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:05.562160 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:05.562160 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:05.562160 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:05.562160 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:05.562160 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:05.562160 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:05.563084 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:06:05.563084 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:06:05.563084 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:05.563084 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:05.563084 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 309/604] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:06:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:05.563084 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194ae4a3-480d-402a-9046-6942596ca479 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:05.640697 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028f662c-3ddd-4353-8ab8-de578bd045a9 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:05.685307 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb24045b-b2ac-44d0-ad2c-fd21c77a0bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:05.686394 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb24045b-b2ac-44d0-ad2c-fd21c77a0bb9 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:05.686683 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb24045b-b2ac-44d0-ad2c-fd21c77a0bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:05.686962 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb24045b-b2ac-44d0-ad2c-fd21c77a0bb9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:05.689018 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194ae4a3-480d-402a-9046-6942596ca479 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:05.707550 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb24045b-b2ac-44d0-ad2c-fd21c77a0bb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:06:05.718291 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-cb24045b-b2ac-44d0-ad2c-fd21c77a0bb9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:06:05.718291 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:05.718291 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:05.718291 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:05.718291 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:05.718291 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:05.718291 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:05.718291 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:05.718291 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:05.718291 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:05.718291 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:05.718291 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:05.718291 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:06:05.718291 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:06:05.718291 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:06:05.718291 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:06:05.719773 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:06:05.719773 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:06:05.719773 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:06:05.719773 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:06:05.719773 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:05.719773 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:05.719773 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:05.719773 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:05.719773 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:05.719773 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:05.719773 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:05.719773 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:05.719773 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:05.719773 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:05.719773 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:05.719773 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:05.720682 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:05.720682 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:05.720682 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:05.720682 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:05.720682 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:06:05.720682 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:06:05.720682 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:06:05.720682 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:06:05.720682 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:06:05.720682 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:06:05.720682 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:06:05.720682 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:05.724467 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 296/605] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jun 21 18:06:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:05.725611 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb24045b-b2ac-44d0-ad2c-fd21c77a0bb9 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:05.737294 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e011f46d-686b-4b25-8685-8b0425ab4b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:05.739092 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e011f46d-686b-4b25-8685-8b0425ab4b80 None admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:05.739092 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e011f46d-686b-4b25-8685-8b0425ab4b80 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:05.739538 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e011f46d-686b-4b25-8685-8b0425ab4b80 None admin] PATH_INFO: `/v3/projects/2a4699d1ec70456da5f8e0aa4b1439b7/users/9672794bc1ca4dcd8259d51d9290384b/roles/acf4e26e0ceb41ddad002dd4edf6373c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:05.763429 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e011f46d-686b-4b25-8685-8b0425ab4b80 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a4699d1ec70456da5f8e0aa4b1439b7, user_id=9672794bc1ca4dcd8259d51d9290384b, role_id=acf4e26e0ceb41ddad002dd4edf6373c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:05.765597 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e011f46d-686b-4b25-8685-8b0425ab4b80 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:05.766343 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:05.771567 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:05.861049 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb24045b-b2ac-44d0-ad2c-fd21c77a0bb9 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:05.903756 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb4ca7e-f8fd-4f33-93b3-93ac1280402f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:05.904516 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb4ca7e-f8fd-4f33-93b3-93ac1280402f None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:05.904807 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb4ca7e-f8fd-4f33-93b3-93ac1280402f None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:05.905086 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb4ca7e-f8fd-4f33-93b3-93ac1280402f None admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:05.905818 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb4ca7e-f8fd-4f33-93b3-93ac1280402f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:05.906159 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb4ca7e-f8fd-4f33-93b3-93ac1280402f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:05.907615 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb4ca7e-f8fd-4f33-93b3-93ac1280402f None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:05.925103 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 297/606] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:06:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:05.925103 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb4ca7e-f8fd-4f33-93b3-93ac1280402f None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:05.942507 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 310/607] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jun 21 18:06:05 2022] PUT /identity/v3/projects/2a4699d1ec70456da5f8e0aa4b1439b7/users/9672794bc1ca4dcd8259d51d9290384b/roles/acf4e26e0ceb41ddad002dd4edf6373c => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:06:05.943789 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e011f46d-686b-4b25-8685-8b0425ab4b80 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:06.136523 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:06.163172 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb4ca7e-f8fd-4f33-93b3-93ac1280402f None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:06.167634 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e011f46d-686b-4b25-8685-8b0425ab4b80 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:06.211363 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46e7210-ead8-4aa5-b5b6-7f8cd50de839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:06.212480 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e7210-ead8-4aa5-b5b6-7f8cd50de839 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:06.212884 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e7210-ead8-4aa5-b5b6-7f8cd50de839 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:06.213203 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e7210-ead8-4aa5-b5b6-7f8cd50de839 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:06.214154 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585f2912-bc9e-4ee4-a56a-08766ea371b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:06.215024 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f2912-bc9e-4ee4-a56a-08766ea371b2 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:06.215776 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f2912-bc9e-4ee4-a56a-08766ea371b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:06.216597 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f2912-bc9e-4ee4-a56a-08766ea371b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:06.217392 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585f2912-bc9e-4ee4-a56a-08766ea371b2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:06.217705 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585f2912-bc9e-4ee4-a56a-08766ea371b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:06.219140 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585f2912-bc9e-4ee4-a56a-08766ea371b2 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:06.225200 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46e7210-ead8-4aa5-b5b6-7f8cd50de839 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:06.226666 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46e7210-ead8-4aa5-b5b6-7f8cd50de839 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:06.236937 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 298/608] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jun 21 18:06:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:06.246815 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585f2912-bc9e-4ee4-a56a-08766ea371b2 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:06.248076 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-a46e7210-ead8-4aa5-b5b6-7f8cd50de839 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:06:06.248076 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:06.248076 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:06.248076 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:06.248076 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:06.248076 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:06.248076 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:06.248076 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:06.248076 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:06.248076 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:06.248076 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:06.248076 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:06.248076 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:06:06.248076 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:06:06.248076 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:06:06.249847 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:06:06.249847 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:06.249847 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:06.249847 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:06.249847 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:06.249847 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:06.249847 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:06.249847 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:06.249847 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:06.249847 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:06.249847 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:06.249847 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:06.249847 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:06.249847 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:06.249847 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:06.249847 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:06.249847 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:06.251759 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:06:06.251759 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:06:06.251759 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:06:06.251759 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:06:06.251759 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:06:06.251759 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:06:06.251759 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:06:06.251759 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:06.251759 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 311/609] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jun 21 18:06:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:06.257294 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46e7210-ead8-4aa5-b5b6-7f8cd50de839 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:06.319447 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:06.380423 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46e7210-ead8-4aa5-b5b6-7f8cd50de839 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:06.383796 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585f2912-bc9e-4ee4-a56a-08766ea371b2 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:06.418199 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb84e7b6-38e2-43a3-8a8e-38bfcb6f5910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:06.419131 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb84e7b6-38e2-43a3-8a8e-38bfcb6f5910 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:06.419577 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb84e7b6-38e2-43a3-8a8e-38bfcb6f5910 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:06.419861 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb84e7b6-38e2-43a3-8a8e-38bfcb6f5910 None admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:06.420710 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb84e7b6-38e2-43a3-8a8e-38bfcb6f5910 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:06.421028 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb84e7b6-38e2-43a3-8a8e-38bfcb6f5910 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:06.422472 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb84e7b6-38e2-43a3-8a8e-38bfcb6f5910 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:06.426986 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750ef727-1f73-4c16-b3e3-c8abe6a910bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:06.427827 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ef727-1f73-4c16-b3e3-c8abe6a910bd None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:06.428115 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ef727-1f73-4c16-b3e3-c8abe6a910bd None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:06.428409 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ef727-1f73-4c16-b3e3-c8abe6a910bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:06.436924 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 312/610] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jun 21 18:06:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:06:06.438374 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a5f689-5caa-4c9e-b034-d28709b23f0c None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:06.438673 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a5f689-5caa-4c9e-b034-d28709b23f0c None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:06.438954 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a5f689-5caa-4c9e-b034-d28709b23f0c None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:06.444383 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 313/611] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:06:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:06:06.445409 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a5f689-5caa-4c9e-b034-d28709b23f0c None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:06.445638 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750ef727-1f73-4c16-b3e3-c8abe6a910bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:06.447865 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750ef727-1f73-4c16-b3e3-c8abe6a910bd None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:06.467921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-750ef727-1f73-4c16-b3e3-c8abe6a910bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:06.467921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:06.467921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:06.467921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:06.467921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:06.467921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:06.467921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:06.467921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:06.467921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:06.467921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:06.467921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:06.467921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:06.467921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:06:06.467921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:06:06.467921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:06:06.468972 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:06:06.468972 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:06.468972 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:06.468972 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:06.468972 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:06.468972 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:06.468972 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:06.468972 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:06.468972 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:06.468972 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:06.468972 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:06.468972 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:06.468972 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:06.468972 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:06.468972 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:06.468972 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:06.468972 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:06.469903 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:06:06.469903 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:06:06.469903 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:06.469903 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:06.470945 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:06.474444 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 299/612] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:06:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:06.476090 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe83a87-3c9a-45d5-8d86-03b0c624296c None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:06.476503 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe83a87-3c9a-45d5-8d86-03b0c624296c None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:06.476961 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe83a87-3c9a-45d5-8d86-03b0c624296c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:06.497833 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-cbe83a87-3c9a-45d5-8d86-03b0c624296c None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:06:06.499199 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-cbe83a87-3c9a-45d5-8d86-03b0c624296c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:06:06.558511 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a5f689-5caa-4c9e-b034-d28709b23f0c None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:06.575639 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 300/613] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:06:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:06:06.576651 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe83a87-3c9a-45d5-8d86-03b0c624296c None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:06.583293 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:06.625985 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a76ead-69b2-455a-b4f8-0356afd39406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:06.627671 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a76ead-69b2-455a-b4f8-0356afd39406 None admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:06.627903 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a76ead-69b2-455a-b4f8-0356afd39406 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:06.628506 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a76ead-69b2-455a-b4f8-0356afd39406 None admin] PATH_INFO: `/v3/projects/2a4699d1ec70456da5f8e0aa4b1439b7/users/59c7f07e58ba4db9a7d402d8f3da1e89/roles/acf4e26e0ceb41ddad002dd4edf6373c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:06.645859 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a76ead-69b2-455a-b4f8-0356afd39406 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a4699d1ec70456da5f8e0aa4b1439b7, user_id=59c7f07e58ba4db9a7d402d8f3da1e89, role_id=acf4e26e0ceb41ddad002dd4edf6373c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:06.647649 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a76ead-69b2-455a-b4f8-0356afd39406 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:06.684411 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe83a87-3c9a-45d5-8d86-03b0c624296c None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:06.714105 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:06.719834 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b8e140-9903-4d9b-a3f4-862ce0cbb806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:06.720549 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b8e140-9903-4d9b-a3f4-862ce0cbb806 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:06.720821 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b8e140-9903-4d9b-a3f4-862ce0cbb806 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:06.721097 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b8e140-9903-4d9b-a3f4-862ce0cbb806 None admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:06.722185 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b8e140-9903-4d9b-a3f4-862ce0cbb806 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:06.722316 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b8e140-9903-4d9b-a3f4-862ce0cbb806 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:06.723861 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b8e140-9903-4d9b-a3f4-862ce0cbb806 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:06.732826 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 314/614] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jun 21 18:06:06 2022] PUT /identity/v3/projects/2a4699d1ec70456da5f8e0aa4b1439b7/users/59c7f07e58ba4db9a7d402d8f3da1e89/roles/acf4e26e0ceb41ddad002dd4edf6373c => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:06:06.734174 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a76ead-69b2-455a-b4f8-0356afd39406 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:06.736730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 301/615] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:06:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:06.744448 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b8e140-9903-4d9b-a3f4-862ce0cbb806 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:06.936781 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b8e140-9903-4d9b-a3f4-862ce0cbb806 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:06.963395 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a76ead-69b2-455a-b4f8-0356afd39406 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:06.979227 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898d945f-9108-4c64-b6db-210f696a62d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:06.980323 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898d945f-9108-4c64-b6db-210f696a62d2 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:06.980776 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898d945f-9108-4c64-b6db-210f696a62d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:06.981196 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898d945f-9108-4c64-b6db-210f696a62d2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:06.990288 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898d945f-9108-4c64-b6db-210f696a62d2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:06.992002 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898d945f-9108-4c64-b6db-210f696a62d2 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:07.005824 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfe0c13-e251-4c67-b238-b0cd042aa6c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:07.006638 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe0c13-e251-4c67-b238-b0cd042aa6c6 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:07.006933 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe0c13-e251-4c67-b238-b0cd042aa6c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:07.007281 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe0c13-e251-4c67-b238-b0cd042aa6c6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:07.009969 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-898d945f-9108-4c64-b6db-210f696a62d2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 21 18:06:07.009969 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:07.009969 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:07.009969 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:07.009969 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:07.009969 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:07.009969 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:07.009969 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:07.009969 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:07.009969 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:07.009969 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:07.009969 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:07.009969 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:06:07.009969 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:06:07.009969 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:06:07.009969 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:06:07.010921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:07.010921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:07.010921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:06:07.010921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:06:07.010921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:06:07.010921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:06:07.010921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:07.010921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:07.010921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:07.010921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:07.010921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:07.010921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:07.010921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:07.010921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:07.010921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:07.010921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:07.010921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:07.010921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:07.011870 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:07.011870 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:07.011870 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:06:07.011870 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:06:07.011870 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:06:07.011870 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:06:07.011870 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:06:07.011870 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:06:07.011870 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 21 18:06:07.011870 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:07.014937 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 302/616] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jun 21 18:06:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:07.020866 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898d945f-9108-4c64-b6db-210f696a62d2 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:07.035422 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfe0c13-e251-4c67-b238-b0cd042aa6c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:06:07.044227 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-7dfe0c13-e251-4c67-b238-b0cd042aa6c6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:06:07.044227 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:07.044227 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:07.044227 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:07.044227 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:07.044227 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:07.044227 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:07.044227 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:07.044227 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:07.044227 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:07.044227 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:07.044227 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:07.044227 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:06:07.044227 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:06:07.044227 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:06:07.044227 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:06:07.045319 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:06:07.045319 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:06:07.045319 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:06:07.045319 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:06:07.045319 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:07.045319 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:07.045319 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:07.045319 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:07.045319 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:07.045319 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:07.045319 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:07.045319 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:07.045319 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:07.045319 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:07.045319 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:07.045319 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:07.046180 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:07.046180 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:07.046180 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:07.046180 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:07.046180 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:06:07.046180 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:06:07.046180 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:06:07.046180 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:06:07.046180 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:06:07.046180 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:06:07.046180 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:06:07.046180 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:07.052930 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 315/617] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jun 21 18:06:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:07.059599 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfe0c13-e251-4c67-b238-b0cd042aa6c6 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:07.132812 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:07.148646 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:07.241182 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898d945f-9108-4c64-b6db-210f696a62d2 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:07.245062 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfe0c13-e251-4c67-b238-b0cd042aa6c6 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:07.294424 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e87996f-c0a2-4c3f-9872-13e6c59d1f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:07.295114 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87996f-c0a2-4c3f-9872-13e6c59d1f09 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:07.295400 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87996f-c0a2-4c3f-9872-13e6c59d1f09 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:07.295683 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87996f-c0a2-4c3f-9872-13e6c59d1f09 None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:07.296517 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e87996f-c0a2-4c3f-9872-13e6c59d1f09 None admin] RBAC: Adding query filter params (name=service) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:07.296831 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e87996f-c0a2-4c3f-9872-13e6c59d1f09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:07.298205 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e87996f-c0a2-4c3f-9872-13e6c59d1f09 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:07.304483 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3006c8f3-494e-4495-83b5-1e36e4054e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:07.305179 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006c8f3-494e-4495-83b5-1e36e4054e92 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:07.305463 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006c8f3-494e-4495-83b5-1e36e4054e92 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:07.305740 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006c8f3-494e-4495-83b5-1e36e4054e92 None admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:07.306640 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3006c8f3-494e-4495-83b5-1e36e4054e92 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:07.307064 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3006c8f3-494e-4495-83b5-1e36e4054e92 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:07.312189 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3006c8f3-494e-4495-83b5-1e36e4054e92 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:07.318973 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 316/618] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jun 21 18:06:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:06:07.324928 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e87996f-c0a2-4c3f-9872-13e6c59d1f09 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:07.352678 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 303/619] 10.222.0.23 () {56 vars in 1135 bytes} [Tue Jun 21 18:06:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:07.355199 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3006c8f3-494e-4495-83b5-1e36e4054e92 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:07.458539 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:07.471385 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e87996f-c0a2-4c3f-9872-13e6c59d1f09 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:07.508688 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db692b32-f512-4c75-afb8-a4d46d619b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:07.510033 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db692b32-f512-4c75-afb8-a4d46d619b0a None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:07.510321 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db692b32-f512-4c75-afb8-a4d46d619b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:07.510639 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db692b32-f512-4c75-afb8-a4d46d619b0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:07.522511 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db692b32-f512-4c75-afb8-a4d46d619b0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:07.523973 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db692b32-f512-4c75-afb8-a4d46d619b0a None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:07.547127 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-db692b32-f512-4c75-afb8-a4d46d619b0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:07.547127 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:07.547127 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:07.547127 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:07.547127 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:07.547127 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:07.547127 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:07.547127 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:07.547127 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:07.547127 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:07.547127 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:07.547127 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:07.547127 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:06:07.547127 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:06:07.547127 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:06:07.549234 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:06:07.549234 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:07.549234 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:07.549234 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:07.549234 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:07.549234 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:07.549234 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:07.549234 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:07.549234 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:07.549234 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:07.549234 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:07.549234 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:07.549234 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:07.549234 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:07.549234 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:07.549234 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:07.549234 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:07.551320 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:06:07.551320 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:06:07.551320 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:07.551320 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:07.551831 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3006c8f3-494e-4495-83b5-1e36e4054e92 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:07.556423 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 317/620] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:06:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:07.556423 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db692b32-f512-4c75-afb8-a4d46d619b0a None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:07.570493 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:07.637354 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d2be69-b3de-4257-b581-5c7893d0d062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:07.640167 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2be69-b3de-4257-b581-5c7893d0d062 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:07.640540 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2be69-b3de-4257-b581-5c7893d0d062 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:07.640850 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2be69-b3de-4257-b581-5c7893d0d062 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:07.662736 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d2be69-b3de-4257-b581-5c7893d0d062 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:07.666483 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d2be69-b3de-4257-b581-5c7893d0d062 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:07.684853 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db692b32-f512-4c75-afb8-a4d46d619b0a None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:07.706675 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-67d2be69-b3de-4257-b581-5c7893d0d062 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:07.706675 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:07.706675 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:07.706675 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:07.706675 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:07.706675 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:07.706675 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:07.706675 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:07.706675 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:07.706675 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:07.706675 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:07.706675 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:07.706675 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:06:07.706675 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:06:07.706675 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:06:07.708938 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:06:07.708938 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:07.708938 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:07.708938 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:07.708938 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:07.708938 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:07.708938 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:07.708938 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:07.708938 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:07.708938 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:07.708938 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:07.708938 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:07.708938 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:07.708938 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:07.708938 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:07.708938 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:07.708938 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:07.710782 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:06:07.710782 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:06:07.710782 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:07.710782 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:07.715298 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 304/621] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:06:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:07.718154 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d8b439-b8d5-4c62-bcda-217b03895411 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:07.719013 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d8b439-b8d5-4c62-bcda-217b03895411 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:07.719818 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d8b439-b8d5-4c62-bcda-217b03895411 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:07.722760 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 305/622] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:06:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:06:07.724301 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d8b439-b8d5-4c62-bcda-217b03895411 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:07.734888 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334e3231-d5e7-43d5-bd85-2a2f12632c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:07.735659 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334e3231-d5e7-43d5-bd85-2a2f12632c55 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:07.736010 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334e3231-d5e7-43d5-bd85-2a2f12632c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:07.736351 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334e3231-d5e7-43d5-bd85-2a2f12632c55 None admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:07.737211 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334e3231-d5e7-43d5-bd85-2a2f12632c55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:07.737583 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334e3231-d5e7-43d5-bd85-2a2f12632c55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:07.739049 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334e3231-d5e7-43d5-bd85-2a2f12632c55 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:07.749611 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:07.753472 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 318/623] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:06:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:07.754936 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d0cc7c-34bd-4821-bd53-3fdd369220fd None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:07.755286 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d0cc7c-34bd-4821-bd53-3fdd369220fd None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:07.755612 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d0cc7c-34bd-4821-bd53-3fdd369220fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:07.778325 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-49d0cc7c-34bd-4821-bd53-3fdd369220fd None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:06:07.779582 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-49d0cc7c-34bd-4821-bd53-3fdd369220fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:06:07.896426 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 319/624] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:06:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:06:07.896426 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d0cc7c-34bd-4821-bd53-3fdd369220fd None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:07.939682 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d8b439-b8d5-4c62-bcda-217b03895411 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:08.024477 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a5f038-61d5-4441-8a41-2cde92dbd106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:08.027754 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a5f038-61d5-4441-8a41-2cde92dbd106 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:08.028583 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a5f038-61d5-4441-8a41-2cde92dbd106 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:08.029606 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a5f038-61d5-4441-8a41-2cde92dbd106 None admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:08.032169 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a5f038-61d5-4441-8a41-2cde92dbd106 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:08.033902 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a5f038-61d5-4441-8a41-2cde92dbd106 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:08.037111 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a5f038-61d5-4441-8a41-2cde92dbd106 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:08.047099 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d0cc7c-34bd-4821-bd53-3fdd369220fd None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:08.056089 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 306/625] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:06:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:08.057239 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a5f038-61d5-4441-8a41-2cde92dbd106 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:08.083479 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:08.085219 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:08.094755 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16d6876-22e8-4562-bc1c-1eeb0f5fbcef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:08.095622 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16d6876-22e8-4562-bc1c-1eeb0f5fbcef None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:08.095909 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16d6876-22e8-4562-bc1c-1eeb0f5fbcef None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:08.096182 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16d6876-22e8-4562-bc1c-1eeb0f5fbcef None admin] PATH_INFO: `/v3/users/nova` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:08.106078 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16d6876-22e8-4562-bc1c-1eeb0f5fbcef None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:08.108061 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16d6876-22e8-4562-bc1c-1eeb0f5fbcef None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:08.126858 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-a16d6876-22e8-4562-bc1c-1eeb0f5fbcef None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 21 18:06:08.126858 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:08.126858 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:08.126858 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:08.126858 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:08.126858 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:08.126858 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:08.126858 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:08.126858 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:08.126858 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:08.126858 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:08.126858 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:08.126858 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:06:08.126858 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:06:08.126858 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:06:08.126858 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:06:08.127954 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:08.127954 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:08.127954 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:06:08.127954 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:06:08.127954 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:06:08.127954 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:06:08.127954 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:08.127954 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:08.127954 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:08.127954 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:08.127954 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:08.127954 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:08.127954 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:08.127954 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:08.127954 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:08.127954 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:08.127954 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:08.127954 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:08.128995 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:08.128995 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:08.128995 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:06:08.128995 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:06:08.128995 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:06:08.128995 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:06:08.128995 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:06:08.128995 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:06:08.128995 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 21 18:06:08.128995 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:08.128995 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 320/626] 10.222.0.23 () {56 vars in 1093 bytes} [Tue Jun 21 18:06:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:08.129606 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16d6876-22e8-4562-bc1c-1eeb0f5fbcef None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:08.187181 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a5f038-61d5-4441-8a41-2cde92dbd106 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:08.231962 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fa3eb8-c4bf-47d1-b3f6-33abd00a66e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:08.233272 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fa3eb8-c4bf-47d1-b3f6-33abd00a66e3 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:08.233758 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fa3eb8-c4bf-47d1-b3f6-33abd00a66e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:08.234245 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fa3eb8-c4bf-47d1-b3f6-33abd00a66e3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:08.246454 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16d6876-22e8-4562-bc1c-1eeb0f5fbcef None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:08.247113 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fa3eb8-c4bf-47d1-b3f6-33abd00a66e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:06:08.255083 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-d9fa3eb8-c4bf-47d1-b3f6-33abd00a66e3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:06:08.255083 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:08.255083 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:08.255083 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:08.255083 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:08.255083 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:08.255083 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:08.255083 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:08.255083 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:08.255083 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:08.255083 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:08.255083 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:08.255083 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:06:08.255083 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:06:08.255083 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:06:08.255083 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:06:08.256041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:06:08.256041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:06:08.256041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:06:08.256041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:06:08.256041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:08.256041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:08.256041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:08.256041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:08.256041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:08.256041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:08.256041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:08.256041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:08.256041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:08.256041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:08.256041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:08.256041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:08.256937 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:08.256937 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:08.256937 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:08.256937 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:08.256937 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:06:08.256937 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:06:08.256937 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:06:08.256937 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:06:08.256937 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:06:08.256937 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:06:08.256937 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:06:08.256937 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:08.264737 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 307/627] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jun 21 18:06:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:08.265844 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fa3eb8-c4bf-47d1-b3f6-33abd00a66e3 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:08.285768 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56d341e-5675-4655-a3da-23f5194ea873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:08.286873 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d341e-5675-4655-a3da-23f5194ea873 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:08.287389 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d341e-5675-4655-a3da-23f5194ea873 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:08.287673 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d341e-5675-4655-a3da-23f5194ea873 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:08.296443 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56d341e-5675-4655-a3da-23f5194ea873 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:08.298060 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56d341e-5675-4655-a3da-23f5194ea873 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:08.315725 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-b56d341e-5675-4655-a3da-23f5194ea873 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:06:08.315725 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:08.315725 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:08.315725 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:08.315725 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:08.315725 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:08.315725 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:08.315725 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:08.315725 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:08.315725 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:08.315725 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:08.315725 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:08.315725 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:06:08.315725 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:06:08.315725 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:06:08.316723 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:06:08.316723 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:08.316723 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:08.316723 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:08.316723 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:08.316723 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:08.316723 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:08.316723 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:08.316723 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:08.316723 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:08.316723 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:08.316723 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:08.316723 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:08.316723 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:08.316723 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:08.316723 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:08.316723 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:08.317664 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:06:08.317664 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:06:08.317664 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:06:08.317664 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:06:08.317664 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:06:08.317664 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:06:08.317664 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:06:08.317664 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:08.317664 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 321/628] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jun 21 18:06:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:08.318171 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56d341e-5675-4655-a3da-23f5194ea873 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:08.378402 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:08.383655 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:08.392907 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fa3eb8-c4bf-47d1-b3f6-33abd00a66e3 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:08.428638 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2b7274-4419-4ad2-a08a-8ac7822c0450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:08.429406 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2b7274-4419-4ad2-a08a-8ac7822c0450 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:08.429691 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2b7274-4419-4ad2-a08a-8ac7822c0450 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:08.430006 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2b7274-4419-4ad2-a08a-8ac7822c0450 None admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:08.430792 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2b7274-4419-4ad2-a08a-8ac7822c0450 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:08.431162 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2b7274-4419-4ad2-a08a-8ac7822c0450 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:08.432741 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2b7274-4419-4ad2-a08a-8ac7822c0450 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:08.435959 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56d341e-5675-4655-a3da-23f5194ea873 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:08.451901 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 308/629] 10.222.0.23 () {56 vars in 1133 bytes} [Tue Jun 21 18:06:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:08.452610 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2b7274-4419-4ad2-a08a-8ac7822c0450 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:08.476822 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5912d30-6d22-4311-a8da-a8608fdb30a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:08.477475 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5912d30-6d22-4311-a8da-a8608fdb30a7 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:08.477757 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5912d30-6d22-4311-a8da-a8608fdb30a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:08.478085 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5912d30-6d22-4311-a8da-a8608fdb30a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:08.478835 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5912d30-6d22-4311-a8da-a8608fdb30a7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:08.479141 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5912d30-6d22-4311-a8da-a8608fdb30a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:08.480707 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5912d30-6d22-4311-a8da-a8608fdb30a7 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:08.489134 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 322/630] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jun 21 18:06:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:08.490388 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db138d3f-b7cf-4232-91fb-2b9226641aa1 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:08.490673 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db138d3f-b7cf-4232-91fb-2b9226641aa1 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:08.490949 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db138d3f-b7cf-4232-91fb-2b9226641aa1 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:08.492336 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 323/631] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:06:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:06:08.492896 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db138d3f-b7cf-4232-91fb-2b9226641aa1 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:08.569985 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2b7274-4419-4ad2-a08a-8ac7822c0450 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:08.606733 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db138d3f-b7cf-4232-91fb-2b9226641aa1 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:08.608567 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae188e1d-3f5d-42a7-a140-2d966ee564d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:08.637735 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae188e1d-3f5d-42a7-a140-2d966ee564d5 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:08.637735 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae188e1d-3f5d-42a7-a140-2d966ee564d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:08.637735 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae188e1d-3f5d-42a7-a140-2d966ee564d5 None admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:08.637735 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae188e1d-3f5d-42a7-a140-2d966ee564d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:08.637735 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae188e1d-3f5d-42a7-a140-2d966ee564d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:08.637735 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae188e1d-3f5d-42a7-a140-2d966ee564d5 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:08.638224 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:08.638296 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:08.638371 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 309/632] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jun 21 18:06:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:08.638371 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef398c2-4ad6-453c-ae04-ee6d318a60ac None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:08.638371 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef398c2-4ad6-453c-ae04-ee6d318a60ac None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:08.638371 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef398c2-4ad6-453c-ae04-ee6d318a60ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:08.646369 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-aef398c2-4ad6-453c-ae04-ee6d318a60ac None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:06:08.647331 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e5a173-d730-499f-afc8-353f5a64ebda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:08.647558 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-aef398c2-4ad6-453c-ae04-ee6d318a60ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:06:08.647995 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e5a173-d730-499f-afc8-353f5a64ebda None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:08.648282 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e5a173-d730-499f-afc8-353f5a64ebda None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:08.648590 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e5a173-d730-499f-afc8-353f5a64ebda None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:08.721566 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 310/633] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:06:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:06:08.722441 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef398c2-4ad6-453c-ae04-ee6d318a60ac None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:08.738434 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e5a173-d730-499f-afc8-353f5a64ebda None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:08.740048 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e5a173-d730-499f-afc8-353f5a64ebda None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:08.839161 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef398c2-4ad6-453c-ae04-ee6d318a60ac None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:08.846411 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 324/634] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:06:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:06:08.855489 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e5a173-d730-499f-afc8-353f5a64ebda None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:08.870592 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:08.872044 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:08.878502 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74c777-6e44-4d58-9dbf-9b2b0ba440b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:08.886846 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74c777-6e44-4d58-9dbf-9b2b0ba440b1 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:08.886846 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74c777-6e44-4d58-9dbf-9b2b0ba440b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:08.886846 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74c777-6e44-4d58-9dbf-9b2b0ba440b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:08.893593 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd74c777-6e44-4d58-9dbf-9b2b0ba440b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:08.895075 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd74c777-6e44-4d58-9dbf-9b2b0ba440b1 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:08.913012 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-cd74c777-6e44-4d58-9dbf-9b2b0ba440b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:08.913012 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:08.913012 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:08.913012 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:08.913012 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:08.913012 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:08.913012 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:08.913012 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:08.913012 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:08.913012 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:08.913012 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:08.913012 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:08.913012 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:06:08.913012 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:06:08.913012 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:06:08.914000 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:06:08.914000 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:08.914000 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:08.914000 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:08.914000 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:08.914000 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:08.914000 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:08.914000 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:08.914000 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:08.914000 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:08.914000 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:08.914000 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:08.914000 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:08.914000 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:08.914000 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:08.914000 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:08.914000 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:08.914937 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:06:08.914937 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:06:08.914937 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:08.914937 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:08.914937 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 311/635] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:06:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:08.915879 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74c777-6e44-4d58-9dbf-9b2b0ba440b1 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:08.969635 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e5a173-d730-499f-afc8-353f5a64ebda None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:09.012417 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5220cb6-07f1-4691-b528-d56f0c8ac654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:09.013316 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5220cb6-07f1-4691-b528-d56f0c8ac654 None admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:09.013601 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5220cb6-07f1-4691-b528-d56f0c8ac654 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:09.013873 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5220cb6-07f1-4691-b528-d56f0c8ac654 None admin] PATH_INFO: `/v3/projects/2a4699d1ec70456da5f8e0aa4b1439b7/users/0e0d25565f1543d9a7577288fc09119f/roles/acf4e26e0ceb41ddad002dd4edf6373c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:09.026737 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74c777-6e44-4d58-9dbf-9b2b0ba440b1 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:09.029473 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5220cb6-07f1-4691-b528-d56f0c8ac654 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a4699d1ec70456da5f8e0aa4b1439b7, user_id=0e0d25565f1543d9a7577288fc09119f, role_id=acf4e26e0ceb41ddad002dd4edf6373c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:09.030933 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5220cb6-07f1-4691-b528-d56f0c8ac654 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:09.061900 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b8410a-f28c-4331-8bd6-1c504b9d8362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:09.062729 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b8410a-f28c-4331-8bd6-1c504b9d8362 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:09.063019 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b8410a-f28c-4331-8bd6-1c504b9d8362 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:09.063299 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b8410a-f28c-4331-8bd6-1c504b9d8362 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:09.073632 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b8410a-f28c-4331-8bd6-1c504b9d8362 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:06:09.081375 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-87b8410a-f28c-4331-8bd6-1c504b9d8362 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:06:09.081375 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:09.081375 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:09.081375 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:09.081375 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:09.081375 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:09.081375 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:09.081375 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:09.081375 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:09.081375 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:09.081375 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:09.081375 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:09.081375 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:06:09.081375 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:06:09.081375 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:06:09.081375 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:06:09.082504 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:06:09.082504 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:06:09.082504 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:06:09.082504 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:06:09.082504 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:09.082504 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:09.082504 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:09.082504 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:09.082504 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:09.082504 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:09.082504 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:09.082504 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:09.082504 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:09.082504 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:09.082504 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:09.082504 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:09.083397 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:09.083397 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:09.083397 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:09.083397 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:09.083397 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:06:09.083397 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:06:09.083397 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:06:09.083397 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:06:09.083397 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:06:09.083397 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:06:09.083397 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:06:09.083397 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:09.083397 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 312/636] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jun 21 18:06:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:09.084138 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b8410a-f28c-4331-8bd6-1c504b9d8362 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:09.107311 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 325/637] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jun 21 18:06:08 2022] PUT /identity/v3/projects/2a4699d1ec70456da5f8e0aa4b1439b7/users/0e0d25565f1543d9a7577288fc09119f/roles/acf4e26e0ceb41ddad002dd4edf6373c => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:06:09.108070 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5220cb6-07f1-4691-b528-d56f0c8ac654 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:09.138808 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:09.199108 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:09.212951 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b8410a-f28c-4331-8bd6-1c504b9d8362 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:09.246112 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5220cb6-07f1-4691-b528-d56f0c8ac654 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:09.255815 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6ced6e-10f5-43b8-b34d-f345683e8252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:09.256939 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ced6e-10f5-43b8-b34d-f345683e8252 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:09.257589 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ced6e-10f5-43b8-b34d-f345683e8252 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:09.258270 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ced6e-10f5-43b8-b34d-f345683e8252 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:09.273913 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6ced6e-10f5-43b8-b34d-f345683e8252 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:09.276125 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6ced6e-10f5-43b8-b34d-f345683e8252 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:09.288879 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7778b21-cd69-421d-9dbb-f89049766243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:09.289557 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7778b21-cd69-421d-9dbb-f89049766243 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:09.289837 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7778b21-cd69-421d-9dbb-f89049766243 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:09.290173 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7778b21-cd69-421d-9dbb-f89049766243 None admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:09.290940 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7778b21-cd69-421d-9dbb-f89049766243 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:09.291379 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7778b21-cd69-421d-9dbb-f89049766243 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:09.292798 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7778b21-cd69-421d-9dbb-f89049766243 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:09.301759 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-1e6ced6e-10f5-43b8-b34d-f345683e8252 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:09.301759 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:09.301759 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:09.301759 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:09.301759 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:09.301759 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:09.301759 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:09.301759 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:09.301759 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:09.301759 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:09.301759 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:09.301759 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:09.301759 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:06:09.301759 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:06:09.301759 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:06:09.302921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:06:09.302921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:09.302921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:09.302921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:09.302921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:09.302921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:09.302921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:09.302921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:09.302921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:09.302921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:09.302921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:09.302921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:09.302921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:09.302921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:09.302921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:09.302921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:09.302921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:09.304280 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:06:09.304280 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:06:09.304280 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:09.304280 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:09.305216 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 326/638] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:06:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:09.306882 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 313/639] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:06:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:09.307367 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959941b0-e75d-43eb-9920-e6a70191c68f None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:09.307700 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959941b0-e75d-43eb-9920-e6a70191c68f None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:09.307980 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959941b0-e75d-43eb-9920-e6a70191c68f None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:09.308356 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6ced6e-10f5-43b8-b34d-f345683e8252 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:09.312423 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 327/640] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:06:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:06:09.312775 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959941b0-e75d-43eb-9920-e6a70191c68f None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:09.414504 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:09.425279 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6ced6e-10f5-43b8-b34d-f345683e8252 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:09.443027 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959941b0-e75d-43eb-9920-e6a70191c68f None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:09.467885 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e870e18-44a3-4253-84d1-efc893f17bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:09.468797 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e870e18-44a3-4253-84d1-efc893f17bdd None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:09.469095 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e870e18-44a3-4253-84d1-efc893f17bdd None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:09.469366 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e870e18-44a3-4253-84d1-efc893f17bdd None admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:09.469875 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:09.470461 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e870e18-44a3-4253-84d1-efc893f17bdd None admin] RBAC: Adding query filter params (name=service) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:09.471280 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e870e18-44a3-4253-84d1-efc893f17bdd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:09.472878 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e870e18-44a3-4253-84d1-efc893f17bdd None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:09.485639 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363f647e-1f01-4135-a430-fd45620ffe3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:09.486547 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f647e-1f01-4135-a430-fd45620ffe3b None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:09.486905 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f647e-1f01-4135-a430-fd45620ffe3b None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:09.487183 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f647e-1f01-4135-a430-fd45620ffe3b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:09.489154 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 314/641] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jun 21 18:06:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:09.491542 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d8f34-6f34-4d03-ba14-74c7dde89a57 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:09.491813 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d8f34-6f34-4d03-ba14-74c7dde89a57 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:09.492096 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d8f34-6f34-4d03-ba14-74c7dde89a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:09.496978 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363f647e-1f01-4135-a430-fd45620ffe3b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:09.498703 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363f647e-1f01-4135-a430-fd45620ffe3b None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:09.512912 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-ca5d8f34-6f34-4d03-ba14-74c7dde89a57 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:06:09.514120 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-ca5d8f34-6f34-4d03-ba14-74c7dde89a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:06:09.515396 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-363f647e-1f01-4135-a430-fd45620ffe3b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 21 18:06:09.515396 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:09.515396 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:09.515396 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:09.515396 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:09.515396 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:09.515396 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:09.515396 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:09.515396 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:09.515396 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:09.515396 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:09.515396 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:09.515396 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:06:09.515396 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:06:09.515396 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:06:09.515396 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:06:09.516417 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:09.516417 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:09.516417 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:06:09.516417 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:06:09.516417 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:06:09.516417 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:06:09.516417 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:09.516417 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:09.516417 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:09.516417 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:09.516417 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:09.516417 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:09.516417 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:09.516417 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:09.516417 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:09.516417 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:09.516417 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:09.516417 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:09.517403 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:09.517403 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:09.517403 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:06:09.517403 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:06:09.517403 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:06:09.517403 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:06:09.517403 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:06:09.517403 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:06:09.517403 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 21 18:06:09.517403 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:09.517403 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 328/642] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:06:09 2022] GET /identity/v3/users/cinder => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:06:09.518041 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363f647e-1f01-4135-a430-fd45620ffe3b None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:09.618539 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 315/643] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:06:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:06:09.624631 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d8f34-6f34-4d03-ba14-74c7dde89a57 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:09.647022 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363f647e-1f01-4135-a430-fd45620ffe3b None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:09.686996 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce3cab4-9e9f-4156-bbef-b43ebde4045f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:09.687577 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce3cab4-9e9f-4156-bbef-b43ebde4045f None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:09.687866 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce3cab4-9e9f-4156-bbef-b43ebde4045f None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:09.688142 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce3cab4-9e9f-4156-bbef-b43ebde4045f None admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:09.688936 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce3cab4-9e9f-4156-bbef-b43ebde4045f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:09.689315 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce3cab4-9e9f-4156-bbef-b43ebde4045f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:09.690721 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce3cab4-9e9f-4156-bbef-b43ebde4045f None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:09.709419 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 329/644] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:06:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:09.709419 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce3cab4-9e9f-4156-bbef-b43ebde4045f None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:09.737806 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:09.756724 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d8f34-6f34-4d03-ba14-74c7dde89a57 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:09.763204 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:09.798760 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d769e0-8225-48da-adaa-a2b01ccf3feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:09.799567 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d769e0-8225-48da-adaa-a2b01ccf3feb None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:09.800382 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d769e0-8225-48da-adaa-a2b01ccf3feb None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:09.800772 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d769e0-8225-48da-adaa-a2b01ccf3feb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:09.814608 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d769e0-8225-48da-adaa-a2b01ccf3feb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:09.816241 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d769e0-8225-48da-adaa-a2b01ccf3feb None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:09.831493 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce3cab4-9e9f-4156-bbef-b43ebde4045f None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:09.835642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-91d769e0-8225-48da-adaa-a2b01ccf3feb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:09.835642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:09.835642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:09.835642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:09.835642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:09.835642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:09.835642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:09.835642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:09.835642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:09.835642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:09.835642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:09.835642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:09.835642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:06:09.835642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:06:09.835642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:06:09.836828 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:06:09.836828 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:09.836828 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:09.836828 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:09.836828 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:09.836828 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:09.836828 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:09.836828 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:09.836828 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:09.836828 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:09.836828 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:09.836828 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:09.836828 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:09.836828 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:09.836828 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:09.836828 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:09.836828 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:09.844185 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:06:09.844185 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:06:09.844185 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:09.844185 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:09.844185 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 316/645] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:06:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:09.844185 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d769e0-8225-48da-adaa-a2b01ccf3feb None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:09.873473 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4964fd7a-3e15-4ca1-a64f-41dd52b83dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:09.874222 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4964fd7a-3e15-4ca1-a64f-41dd52b83dad None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:09.874434 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4964fd7a-3e15-4ca1-a64f-41dd52b83dad None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:09.874707 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4964fd7a-3e15-4ca1-a64f-41dd52b83dad None admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:09.875565 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4964fd7a-3e15-4ca1-a64f-41dd52b83dad None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:09.875872 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4964fd7a-3e15-4ca1-a64f-41dd52b83dad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:09.877390 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4964fd7a-3e15-4ca1-a64f-41dd52b83dad None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:09.907063 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 330/646] 10.222.0.23 () {56 vars in 1137 bytes} [Tue Jun 21 18:06:09 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:09.908096 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4964fd7a-3e15-4ca1-a64f-41dd52b83dad None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:09.991704 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d769e0-8225-48da-adaa-a2b01ccf3feb None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:10.041851 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f8642f-8b0f-49e2-834a-504b73de119b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:10.042833 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f8642f-8b0f-49e2-834a-504b73de119b None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:10.043312 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f8642f-8b0f-49e2-834a-504b73de119b None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:10.043733 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f8642f-8b0f-49e2-834a-504b73de119b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:10.047474 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:10.058336 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f8642f-8b0f-49e2-834a-504b73de119b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:06:10.066936 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:10.067708 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-42f8642f-8b0f-49e2-834a-504b73de119b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:06:10.067708 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:10.067708 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:10.067708 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:10.067708 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:10.067708 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:10.067708 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:10.067708 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:10.067708 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:10.067708 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:10.067708 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:10.067708 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:10.067708 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:06:10.067708 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:06:10.067708 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:06:10.067708 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:06:10.068721 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:06:10.068721 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:06:10.068721 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:06:10.068721 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:06:10.068721 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:10.068721 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:10.068721 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:10.068721 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:10.068721 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:10.068721 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:10.068721 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:10.068721 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:10.068721 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:10.068721 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:10.068721 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:10.068721 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:10.069572 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:10.069572 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:10.069572 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:10.069572 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:10.069572 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:06:10.069572 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:06:10.069572 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:06:10.069572 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:06:10.069572 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:06:10.069572 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:06:10.069572 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:06:10.069572 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:10.069572 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 317/647] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jun 21 18:06:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:10.070283 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f8642f-8b0f-49e2-834a-504b73de119b None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:10.074309 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4964fd7a-3e15-4ca1-a64f-41dd52b83dad None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:10.116478 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e064896f-7a53-4ca2-bbea-267e492e527d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:10.117380 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e064896f-7a53-4ca2-bbea-267e492e527d None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:10.117730 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e064896f-7a53-4ca2-bbea-267e492e527d None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:10.118073 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e064896f-7a53-4ca2-bbea-267e492e527d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:10.127322 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e064896f-7a53-4ca2-bbea-267e492e527d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:10.129426 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e064896f-7a53-4ca2-bbea-267e492e527d None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:10.145849 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-e064896f-7a53-4ca2-bbea-267e492e527d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:06:10.145849 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:10.145849 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:10.145849 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:10.145849 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:10.145849 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:10.145849 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:10.145849 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:10.145849 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:10.145849 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:10.145849 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:10.145849 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:10.145849 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:06:10.145849 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:06:10.145849 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:06:10.147043 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:06:10.147043 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:10.147043 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:10.147043 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:10.147043 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:10.147043 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:10.147043 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:10.147043 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:10.147043 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:10.147043 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:10.147043 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:10.147043 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:10.147043 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:10.147043 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:10.147043 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:10.147043 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:10.147043 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:10.147967 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:06:10.147967 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:06:10.147967 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:06:10.147967 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:06:10.147967 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:06:10.147967 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:06:10.147967 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:06:10.147967 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:10.147967 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 331/648] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jun 21 18:06:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:10.147967 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e064896f-7a53-4ca2-bbea-267e492e527d None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:10.207531 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f8642f-8b0f-49e2-834a-504b73de119b None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:10.252716 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc769f6-9f18-4ac6-931b-5bf4ea654b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:10.253523 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc769f6-9f18-4ac6-931b-5bf4ea654b60 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:10.253908 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc769f6-9f18-4ac6-931b-5bf4ea654b60 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:10.254292 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc769f6-9f18-4ac6-931b-5bf4ea654b60 None admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:10.255195 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc769f6-9f18-4ac6-931b-5bf4ea654b60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:10.255636 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc769f6-9f18-4ac6-931b-5bf4ea654b60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:10.257294 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc769f6-9f18-4ac6-931b-5bf4ea654b60 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:10.276899 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 318/649] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:06:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:10.276899 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc769f6-9f18-4ac6-931b-5bf4ea654b60 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:10.298549 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e064896f-7a53-4ca2-bbea-267e492e527d None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:10.337225 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9aac9-84e9-4bb6-9421-3c63dc9cb276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:10.337893 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9aac9-84e9-4bb6-9421-3c63dc9cb276 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:10.338213 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9aac9-84e9-4bb6-9421-3c63dc9cb276 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:10.338499 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9aac9-84e9-4bb6-9421-3c63dc9cb276 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:10.347235 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:10.373953 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:10.398034 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc769f6-9f18-4ac6-931b-5bf4ea654b60 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:10.421627 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e9aac9-84e9-4bb6-9421-3c63dc9cb276 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:10.423110 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e9aac9-84e9-4bb6-9421-3c63dc9cb276 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:10.441257 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0f27b2-fc09-4753-a63a-e04f3ac8949b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:10.442438 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f27b2-fc09-4753-a63a-e04f3ac8949b None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:10.442946 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f27b2-fc09-4753-a63a-e04f3ac8949b None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:10.443457 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f27b2-fc09-4753-a63a-e04f3ac8949b None admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:10.444957 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0f27b2-fc09-4753-a63a-e04f3ac8949b None admin] RBAC: Adding query filter params (name=service) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:10.445562 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0f27b2-fc09-4753-a63a-e04f3ac8949b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:10.448138 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0f27b2-fc09-4753-a63a-e04f3ac8949b None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:10.456839 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 319/650] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jun 21 18:06:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:10.457682 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0f27b2-fc09-4753-a63a-e04f3ac8949b None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:10.539993 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 332/651] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:06:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:06:10.541296 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9aac9-84e9-4bb6-9421-3c63dc9cb276 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:10.598762 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0f27b2-fc09-4753-a63a-e04f3ac8949b None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:10.615799 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:10.619066 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:10.637339 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e93f2-9b66-4670-b3f7-234e1c33f440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:10.638314 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e93f2-9b66-4670-b3f7-234e1c33f440 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:10.638762 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e93f2-9b66-4670-b3f7-234e1c33f440 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:10.639195 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e93f2-9b66-4670-b3f7-234e1c33f440 None admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:10.640133 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3e93f2-9b66-4670-b3f7-234e1c33f440 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:10.640626 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3e93f2-9b66-4670-b3f7-234e1c33f440 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:10.642265 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3e93f2-9b66-4670-b3f7-234e1c33f440 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:10.658259 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 320/652] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jun 21 18:06:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:10.660611 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e93f2-9b66-4670-b3f7-234e1c33f440 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:10.673958 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9aac9-84e9-4bb6-9421-3c63dc9cb276 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:10.715970 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc85180-6e33-4084-844d-5aa908ac5643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:10.716812 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc85180-6e33-4084-844d-5aa908ac5643 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:10.717099 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc85180-6e33-4084-844d-5aa908ac5643 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:10.717377 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc85180-6e33-4084-844d-5aa908ac5643 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:10.727499 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc85180-6e33-4084-844d-5aa908ac5643 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:10.729553 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc85180-6e33-4084-844d-5aa908ac5643 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:10.745152 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-4cc85180-6e33-4084-844d-5aa908ac5643 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 21 18:06:10.745152 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:10.745152 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:10.745152 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:10.745152 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:10.745152 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:10.745152 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:10.745152 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:10.745152 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:10.745152 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:10.745152 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:10.745152 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:10.745152 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:06:10.745152 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:06:10.745152 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:06:10.745152 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:06:10.746931 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:10.746931 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:10.746931 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:06:10.746931 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:06:10.746931 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:06:10.746931 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:06:10.746931 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:10.746931 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:10.746931 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:10.746931 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:10.746931 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:10.746931 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:10.746931 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:10.746931 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:10.746931 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:10.746931 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:10.746931 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:10.746931 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:10.749024 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:10.749024 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:10.749024 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:06:10.749024 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:06:10.749024 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:06:10.749024 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:06:10.749024 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:06:10.749024 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:06:10.749024 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 21 18:06:10.749024 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:10.749024 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 333/653] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:06:10 2022] GET /identity/v3/users/glance => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:10.749024 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc85180-6e33-4084-844d-5aa908ac5643 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:10.795851 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e93f2-9b66-4670-b3f7-234e1c33f440 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:10.833080 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a626a22d-e5aa-4014-b004-af09cd925a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:10.833897 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a626a22d-e5aa-4014-b004-af09cd925a36 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:10.834184 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a626a22d-e5aa-4014-b004-af09cd925a36 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:10.834461 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a626a22d-e5aa-4014-b004-af09cd925a36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:10.857808 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a626a22d-e5aa-4014-b004-af09cd925a36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:10.861074 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a626a22d-e5aa-4014-b004-af09cd925a36 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:10.874708 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc85180-6e33-4084-844d-5aa908ac5643 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:10.887607 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-a626a22d-e5aa-4014-b004-af09cd925a36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:10.887607 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:10.887607 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:10.887607 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:10.887607 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:10.887607 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:10.887607 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:10.887607 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:10.887607 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:10.887607 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:10.887607 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:10.887607 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:10.887607 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:06:10.887607 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:06:10.887607 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:06:10.890074 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:06:10.890074 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:10.890074 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:10.890074 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:10.890074 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:10.890074 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:10.890074 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:10.890074 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:10.890074 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:10.890074 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:10.890074 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:10.890074 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:10.890074 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:10.890074 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:10.890074 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:10.890074 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:10.890074 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:10.891894 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:06:10.891894 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:06:10.891894 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:10.891894 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:10.891894 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 321/654] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:06:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:10.891894 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a626a22d-e5aa-4014-b004-af09cd925a36 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:10.910095 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:10.925366 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c2ef20-d4ff-43ae-a76a-bb48f8541dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:10.926185 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c2ef20-d4ff-43ae-a76a-bb48f8541dc4 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:10.926633 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c2ef20-d4ff-43ae-a76a-bb48f8541dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:10.926964 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c2ef20-d4ff-43ae-a76a-bb48f8541dc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:10.932154 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:10.941149 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c2ef20-d4ff-43ae-a76a-bb48f8541dc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:10.943085 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c2ef20-d4ff-43ae-a76a-bb48f8541dc4 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:10.962589 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-a0c2ef20-d4ff-43ae-a76a-bb48f8541dc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:10.962589 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:10.962589 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:10.962589 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:10.962589 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:10.962589 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:10.962589 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:10.962589 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:10.962589 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:10.962589 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:10.962589 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:10.962589 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:10.962589 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:06:10.962589 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:06:10.962589 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:06:10.963823 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:06:10.963823 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:10.963823 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:10.963823 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:10.963823 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:10.963823 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:10.963823 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:10.963823 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:10.963823 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:10.963823 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:10.963823 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:10.963823 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:10.963823 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:10.963823 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:10.963823 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:10.963823 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:10.963823 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:10.964968 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:06:10.964968 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:06:10.964968 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:10.964968 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:10.965564 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 334/655] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:06:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:10.966382 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c2ef20-d4ff-43ae-a76a-bb48f8541dc4 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:11.020351 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a626a22d-e5aa-4014-b004-af09cd925a36 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:11.058352 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6e056e-8575-4072-a233-7e7be84d01a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:11.058936 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e056e-8575-4072-a233-7e7be84d01a3 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:11.059215 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e056e-8575-4072-a233-7e7be84d01a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:11.059492 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e056e-8575-4072-a233-7e7be84d01a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:11.060312 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6e056e-8575-4072-a233-7e7be84d01a3 None admin] RBAC: Adding query filter params (role.id=acf4e26e0ceb41ddad002dd4edf6373c, scope.project.id=2a4699d1ec70456da5f8e0aa4b1439b7, user.id=904255dd3c1f406eb10f6fc65f9cd098) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:11.060647 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6e056e-8575-4072-a233-7e7be84d01a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:11.062073 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6e056e-8575-4072-a233-7e7be84d01a3 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:11.076422 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 322/656] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jun 21 18:06:10 2022] GET /identity/v3/role_assignments?user.id=904255dd3c1f406eb10f6fc65f9cd098&scope.project.id=2a4699d1ec70456da5f8e0aa4b1439b7&role.id=acf4e26e0ceb41ddad002dd4edf6373c => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:11.076422 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6e056e-8575-4072-a233-7e7be84d01a3 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:11.086052 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c2ef20-d4ff-43ae-a76a-bb48f8541dc4 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:11.139849 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131788d5-6f1d-402a-9e8a-73e4fb7ab6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:11.140935 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131788d5-6f1d-402a-9e8a-73e4fb7ab6ff None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:11.141300 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131788d5-6f1d-402a-9e8a-73e4fb7ab6ff None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:11.142146 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131788d5-6f1d-402a-9e8a-73e4fb7ab6ff None admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:11.143372 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131788d5-6f1d-402a-9e8a-73e4fb7ab6ff None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:11.143758 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131788d5-6f1d-402a-9e8a-73e4fb7ab6ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:11.146317 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131788d5-6f1d-402a-9e8a-73e4fb7ab6ff None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:11.174616 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 335/657] 10.222.0.23 () {56 vars in 1137 bytes} [Tue Jun 21 18:06:10 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:11.175616 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131788d5-6f1d-402a-9e8a-73e4fb7ab6ff None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:11.189411 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:11.197916 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6e056e-8575-4072-a233-7e7be84d01a3 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:11.239280 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:11.247491 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c7d06f-1a59-4732-9843-bcf26dcda03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:11.248268 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c7d06f-1a59-4732-9843-bcf26dcda03c None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:11.248630 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c7d06f-1a59-4732-9843-bcf26dcda03c None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:11.248946 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c7d06f-1a59-4732-9843-bcf26dcda03c None admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:11.251297 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c7d06f-1a59-4732-9843-bcf26dcda03c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:11.251297 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c7d06f-1a59-4732-9843-bcf26dcda03c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:11.253231 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c7d06f-1a59-4732-9843-bcf26dcda03c None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:11.268661 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 323/658] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:06:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:11.269624 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c7d06f-1a59-4732-9843-bcf26dcda03c None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:11.309228 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131788d5-6f1d-402a-9e8a-73e4fb7ab6ff None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:11.352227 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4eac42-62b2-4105-bb9a-6619905cb2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:11.353495 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4eac42-62b2-4105-bb9a-6619905cb2ec None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:11.353774 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4eac42-62b2-4105-bb9a-6619905cb2ec None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:11.354050 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4eac42-62b2-4105-bb9a-6619905cb2ec None admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:11.354781 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4eac42-62b2-4105-bb9a-6619905cb2ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:11.355085 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4eac42-62b2-4105-bb9a-6619905cb2ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:11.356506 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4eac42-62b2-4105-bb9a-6619905cb2ec None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:11.372490 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 336/659] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:06:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:11.372490 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f4cdf-6336-45ff-9d8b-9a6a5f6cbd81 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:11.372490 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f4cdf-6336-45ff-9d8b-9a6a5f6cbd81 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:11.372490 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f4cdf-6336-45ff-9d8b-9a6a5f6cbd81 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:11.372919 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 337/660] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:06:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:06:11.373703 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610f4cdf-6336-45ff-9d8b-9a6a5f6cbd81 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:11.391618 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c7d06f-1a59-4732-9843-bcf26dcda03c None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:11.428718 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef42db60-9df7-48f5-b351-cacda3d5874f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:11.429640 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef42db60-9df7-48f5-b351-cacda3d5874f None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:11.430105 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef42db60-9df7-48f5-b351-cacda3d5874f None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:11.430576 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef42db60-9df7-48f5-b351-cacda3d5874f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:11.459369 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:11.475507 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:11.489906 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610f4cdf-6336-45ff-9d8b-9a6a5f6cbd81 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:11.508380 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef42db60-9df7-48f5-b351-cacda3d5874f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:11.511110 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef42db60-9df7-48f5-b351-cacda3d5874f None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:11.520787 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4251bd-c68f-4246-9c50-dada3467f11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:11.521586 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4251bd-c68f-4246-9c50-dada3467f11b None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:11.521886 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4251bd-c68f-4246-9c50-dada3467f11b None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:11.522162 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4251bd-c68f-4246-9c50-dada3467f11b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:11.529463 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4251bd-c68f-4246-9c50-dada3467f11b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:11.530930 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4251bd-c68f-4246-9c50-dada3467f11b None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:11.546892 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-ba4251bd-c68f-4246-9c50-dada3467f11b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 21 18:06:11.546892 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:11.546892 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:11.546892 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:11.546892 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:11.546892 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:11.546892 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:11.546892 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:11.546892 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:11.546892 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:11.546892 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:11.546892 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:11.546892 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:06:11.546892 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:06:11.546892 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:06:11.546892 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:06:11.547998 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:11.547998 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:11.547998 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:06:11.547998 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:06:11.547998 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:06:11.547998 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:06:11.547998 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:11.547998 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:11.547998 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:11.547998 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:11.547998 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:11.547998 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:11.547998 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:11.547998 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:11.547998 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:11.547998 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:11.547998 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:11.547998 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:11.549601 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:11.549601 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:11.549601 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:06:11.549601 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:06:11.549601 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:06:11.549601 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:06:11.549601 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:06:11.549601 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:06:11.549601 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 21 18:06:11.549601 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:11.549601 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 338/661] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jun 21 18:06:11 2022] GET /identity/v3/users/neutron => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:11.549601 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4251bd-c68f-4246-9c50-dada3467f11b None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:11.625650 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 324/662] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:06:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:06:11.626905 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee6ed8-65d3-40c6-af35-169eeac018fd None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:11.627235 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee6ed8-65d3-40c6-af35-169eeac018fd None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:11.627550 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee6ed8-65d3-40c6-af35-169eeac018fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:11.644961 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-28ee6ed8-65d3-40c6-af35-169eeac018fd None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:06:11.646154 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-28ee6ed8-65d3-40c6-af35-169eeac018fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:06:11.657644 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4251bd-c68f-4246-9c50-dada3467f11b None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:11.691359 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c151983b-ea75-4cdb-92ee-e8697d2eb25e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:11.692162 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c151983b-ea75-4cdb-92ee-e8697d2eb25e None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:11.692477 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c151983b-ea75-4cdb-92ee-e8697d2eb25e None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:11.692759 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c151983b-ea75-4cdb-92ee-e8697d2eb25e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:11.700320 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:11.700504 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c151983b-ea75-4cdb-92ee-e8697d2eb25e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:11.701935 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c151983b-ea75-4cdb-92ee-e8697d2eb25e None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:11.716487 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-c151983b-ea75-4cdb-92ee-e8697d2eb25e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:06:11.716487 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:11.716487 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:11.716487 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:11.716487 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:11.716487 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:11.716487 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:11.716487 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:11.716487 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:11.716487 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:11.716487 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:11.716487 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:11.716487 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:06:11.716487 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:06:11.716487 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:06:11.722596 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:06:11.722596 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:11.722596 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:11.722596 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:11.722596 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:11.722596 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:11.722596 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:11.722596 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:11.722596 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:11.722596 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:11.722596 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:11.722596 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:11.722596 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:11.722596 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:11.722596 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:11.722596 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:11.722596 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:11.724249 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:06:11.724249 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:06:11.724249 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:06:11.724249 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:06:11.724249 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:06:11.724249 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:06:11.724249 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:06:11.724249 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:11.724249 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 339/663] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jun 21 18:06:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:11.724249 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c151983b-ea75-4cdb-92ee-e8697d2eb25e None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:11.725032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 325/664] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:06:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:06:11.725032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ee6ed8-65d3-40c6-af35-169eeac018fd None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:11.743259 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:11.831767 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ee6ed8-65d3-40c6-af35-169eeac018fd None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:11.838602 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c151983b-ea75-4cdb-92ee-e8697d2eb25e None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:11.867773 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe1c244-6847-42c5-93d0-d285cce4a08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:11.869166 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe1c244-6847-42c5-93d0-d285cce4a08c None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:11.869493 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe1c244-6847-42c5-93d0-d285cce4a08c None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:11.869804 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe1c244-6847-42c5-93d0-d285cce4a08c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:11.880579 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe1c244-6847-42c5-93d0-d285cce4a08c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:11.880837 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c524fcb5-acd8-4e6e-a856-73382e785f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:11.883816 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe1c244-6847-42c5-93d0-d285cce4a08c None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:11.884715 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c524fcb5-acd8-4e6e-a856-73382e785f45 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:11.885109 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c524fcb5-acd8-4e6e-a856-73382e785f45 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:11.885493 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c524fcb5-acd8-4e6e-a856-73382e785f45 None admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:11.886855 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c524fcb5-acd8-4e6e-a856-73382e785f45 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:11.887249 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c524fcb5-acd8-4e6e-a856-73382e785f45 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:11.889288 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c524fcb5-acd8-4e6e-a856-73382e785f45 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:11.904119 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-7fe1c244-6847-42c5-93d0-d285cce4a08c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:11.904119 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:11.904119 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:11.904119 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:11.904119 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:11.904119 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:11.904119 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:11.904119 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:11.904119 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:11.904119 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:11.904119 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:11.904119 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:11.904119 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:06:11.904119 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:06:11.904119 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:06:11.905242 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:06:11.905242 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:11.905242 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:11.905242 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:11.905242 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:11.905242 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:11.905242 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:11.905242 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:11.905242 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:11.905242 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:11.905242 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:11.905242 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:11.905242 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:11.905242 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:11.905242 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:11.905242 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:11.905242 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:11.906399 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:06:11.906399 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:06:11.906399 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:11.906399 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:11.907820 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 326/665] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:06:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:11.908974 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe1c244-6847-42c5-93d0-d285cce4a08c None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:11.916023 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 340/666] 10.222.0.23 () {56 vars in 1139 bytes} [Tue Jun 21 18:06:11 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:11.916023 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c524fcb5-acd8-4e6e-a856-73382e785f45 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:11.950511 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:11.992329 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:12.019569 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe1c244-6847-42c5-93d0-d285cce4a08c None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:12.028325 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c524fcb5-acd8-4e6e-a856-73382e785f45 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:12.060633 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a79f1c1-43fd-4a22-9b78-c97b4547cac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:12.061385 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a79f1c1-43fd-4a22-9b78-c97b4547cac7 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:12.061671 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a79f1c1-43fd-4a22-9b78-c97b4547cac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:12.061947 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a79f1c1-43fd-4a22-9b78-c97b4547cac7 None admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:12.062733 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a79f1c1-43fd-4a22-9b78-c97b4547cac7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:12.063042 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a79f1c1-43fd-4a22-9b78-c97b4547cac7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:12.064495 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a79f1c1-43fd-4a22-9b78-c97b4547cac7 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:12.070479 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4ca792-56ce-4f70-aef4-59ac29691134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:12.071289 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ca792-56ce-4f70-aef4-59ac29691134 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:12.072129 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ca792-56ce-4f70-aef4-59ac29691134 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:12.072529 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ca792-56ce-4f70-aef4-59ac29691134 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:12.077517 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 327/667] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jun 21 18:06:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:12.078342 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a79f1c1-43fd-4a22-9b78-c97b4547cac7 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:12.090929 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4ca792-56ce-4f70-aef4-59ac29691134 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:06:12.099070 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-5a4ca792-56ce-4f70-aef4-59ac29691134 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:06:12.099070 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:12.099070 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:12.099070 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:12.099070 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:12.099070 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:12.099070 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:12.099070 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:12.099070 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:12.099070 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:12.099070 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:12.099070 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:12.099070 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:06:12.099070 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:06:12.099070 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:06:12.099070 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:06:12.100032 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:06:12.100032 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:06:12.100032 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:06:12.100032 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:06:12.100032 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:12.100032 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:12.100032 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:12.100032 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:12.100032 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:12.100032 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:12.100032 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:12.100032 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:12.100032 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:12.100032 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:12.100032 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:12.100032 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:12.101545 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:12.101545 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:12.101545 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:12.101545 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:12.101545 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:06:12.101545 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:06:12.101545 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:06:12.101545 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:06:12.101545 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:06:12.101545 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:06:12.101545 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:06:12.101545 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:12.101545 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 341/668] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jun 21 18:06:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:12.101545 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4ca792-56ce-4f70-aef4-59ac29691134 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:12.188664 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:12.194315 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a79f1c1-43fd-4a22-9b78-c97b4547cac7 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:12.209635 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4ca792-56ce-4f70-aef4-59ac29691134 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:12.232108 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd2dce6-cf0c-490e-bf29-0abf5d550ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:12.232887 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd2dce6-cf0c-490e-bf29-0abf5d550ac3 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:12.233174 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd2dce6-cf0c-490e-bf29-0abf5d550ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:12.233455 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd2dce6-cf0c-490e-bf29-0abf5d550ac3 None admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:12.234179 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd2dce6-cf0c-490e-bf29-0abf5d550ac3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:12.234485 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd2dce6-cf0c-490e-bf29-0abf5d550ac3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:12.235869 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd2dce6-cf0c-490e-bf29-0abf5d550ac3 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:12.239257 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:12.247604 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8d2d0b-9c81-43cf-8153-c9de11948cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:12.248260 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8d2d0b-9c81-43cf-8153-c9de11948cec None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:12.248588 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8d2d0b-9c81-43cf-8153-c9de11948cec None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:12.248866 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8d2d0b-9c81-43cf-8153-c9de11948cec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:12.249425 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 328/669] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:06:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:12.250062 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd2dce6-cf0c-490e-bf29-0abf5d550ac3 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:12.333232 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8d2d0b-9c81-43cf-8153-c9de11948cec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:12.334902 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8d2d0b-9c81-43cf-8153-c9de11948cec None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:12.363406 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd2dce6-cf0c-490e-bf29-0abf5d550ac3 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:12.401943 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b872619f-5a3e-43df-a672-dbc5c30e2606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:12.402623 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b872619f-5a3e-43df-a672-dbc5c30e2606 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:12.402910 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b872619f-5a3e-43df-a672-dbc5c30e2606 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:12.403192 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b872619f-5a3e-43df-a672-dbc5c30e2606 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:12.404026 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b872619f-5a3e-43df-a672-dbc5c30e2606 None admin] RBAC: Adding query filter params (role.id=acf4e26e0ceb41ddad002dd4edf6373c, scope.project.id=2a4699d1ec70456da5f8e0aa4b1439b7, user.id=77c6e43555944902a22ba2fe17a8e99f) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:12.404344 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b872619f-5a3e-43df-a672-dbc5c30e2606 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:12.405806 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b872619f-5a3e-43df-a672-dbc5c30e2606 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:12.420420 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 329/670] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jun 21 18:06:12 2022] GET /identity/v3/role_assignments?user.id=77c6e43555944902a22ba2fe17a8e99f&scope.project.id=2a4699d1ec70456da5f8e0aa4b1439b7&role.id=acf4e26e0ceb41ddad002dd4edf6373c => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:12.420420 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b872619f-5a3e-43df-a672-dbc5c30e2606 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:12.451330 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:12.464500 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 342/671] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:06:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:06:12.465952 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8d2d0b-9c81-43cf-8153-c9de11948cec None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:12.507826 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:12.549570 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b872619f-5a3e-43df-a672-dbc5c30e2606 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:12.590928 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f8526d-ea32-4a2e-b93e-a73864890680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:12.591835 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f8526d-ea32-4a2e-b93e-a73864890680 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:12.592117 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f8526d-ea32-4a2e-b93e-a73864890680 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:12.592460 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f8526d-ea32-4a2e-b93e-a73864890680 None admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:12.593178 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f8526d-ea32-4a2e-b93e-a73864890680 None admin] RBAC: Adding query filter params (name=service) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:12.593483 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f8526d-ea32-4a2e-b93e-a73864890680 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:12.594850 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f8526d-ea32-4a2e-b93e-a73864890680 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:12.605503 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 330/672] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jun 21 18:06:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:12.606466 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f8526d-ea32-4a2e-b93e-a73864890680 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:12.612628 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8d2d0b-9c81-43cf-8153-c9de11948cec None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:12.652533 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2767af1f-63d8-4a12-9057-979a9046b50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:12.653345 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2767af1f-63d8-4a12-9057-979a9046b50d None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:12.653624 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2767af1f-63d8-4a12-9057-979a9046b50d None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:12.653895 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2767af1f-63d8-4a12-9057-979a9046b50d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:12.661568 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2767af1f-63d8-4a12-9057-979a9046b50d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:12.663122 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2767af1f-63d8-4a12-9057-979a9046b50d None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:12.679029 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-2767af1f-63d8-4a12-9057-979a9046b50d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:06:12.679029 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:12.679029 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:12.679029 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:12.679029 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:12.679029 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:12.679029 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:12.679029 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:12.679029 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:12.679029 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:12.679029 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:12.679029 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:12.679029 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:06:12.679029 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:06:12.679029 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:06:12.680111 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:06:12.680111 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:12.680111 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:12.680111 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:12.680111 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:12.680111 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:12.680111 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:12.680111 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:12.680111 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:12.680111 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:12.680111 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:12.680111 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:12.680111 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:12.680111 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:12.680111 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:12.680111 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:12.680111 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:12.681928 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:06:12.681928 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:06:12.681928 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:06:12.681928 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:06:12.681928 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:06:12.681928 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:06:12.681928 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:06:12.681928 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:12.681928 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 343/673] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jun 21 18:06:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:12.681928 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2767af1f-63d8-4a12-9057-979a9046b50d None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:12.708716 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:12.733171 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f8526d-ea32-4a2e-b93e-a73864890680 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:12.773707 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b081e8-ae9c-4fee-a8cf-b21a080507e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:12.774625 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b081e8-ae9c-4fee-a8cf-b21a080507e0 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:12.776081 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b081e8-ae9c-4fee-a8cf-b21a080507e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:12.776365 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b081e8-ae9c-4fee-a8cf-b21a080507e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:12.778476 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:12.788066 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b081e8-ae9c-4fee-a8cf-b21a080507e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:12.794276 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b081e8-ae9c-4fee-a8cf-b21a080507e0 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:12.794409 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2767af1f-63d8-4a12-9057-979a9046b50d None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:12.814552 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-75b081e8-ae9c-4fee-a8cf-b21a080507e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:12.814552 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:12.814552 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:12.814552 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:12.814552 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:12.814552 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:12.814552 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:12.814552 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:12.814552 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:12.814552 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:12.814552 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:12.814552 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:12.814552 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:06:12.814552 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:06:12.814552 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:06:12.815737 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:06:12.815737 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:12.815737 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:12.815737 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:12.815737 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:12.815737 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:12.815737 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:12.815737 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:12.815737 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:12.815737 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:12.815737 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:12.815737 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:12.815737 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:12.815737 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:12.815737 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:12.815737 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:12.815737 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:12.816862 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:06:12.816862 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:06:12.816862 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:12.816862 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:12.816862 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 331/674] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:06:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:12.820751 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b081e8-ae9c-4fee-a8cf-b21a080507e0 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:12.835712 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92c604c-1ddb-45f8-bc08-aff06dc71a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:12.836812 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c604c-1ddb-45f8-bc08-aff06dc71a61 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:12.837945 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c604c-1ddb-45f8-bc08-aff06dc71a61 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:12.838302 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c604c-1ddb-45f8-bc08-aff06dc71a61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:12.853862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92c604c-1ddb-45f8-bc08-aff06dc71a61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:12.855336 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92c604c-1ddb-45f8-bc08-aff06dc71a61 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:12.879172 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-f92c604c-1ddb-45f8-bc08-aff06dc71a61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:12.879172 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:12.879172 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:12.879172 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:12.879172 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:12.879172 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:12.879172 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:12.879172 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:12.879172 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:12.879172 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:12.879172 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:12.879172 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:12.879172 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:06:12.879172 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:06:12.879172 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:06:12.880175 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:06:12.880175 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:12.880175 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:12.880175 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:12.880175 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:12.880175 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:12.880175 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:12.880175 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:12.880175 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:12.880175 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:12.880175 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:12.880175 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:12.880175 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:12.880175 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:12.880175 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:12.880175 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:12.880175 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:12.881145 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:06:12.881145 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:06:12.881145 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:12.881145 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:12.887126 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 344/675] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:06:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:12.888796 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92c604c-1ddb-45f8-bc08-aff06dc71a61 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:12.946295 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b081e8-ae9c-4fee-a8cf-b21a080507e0 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:12.987787 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3836f0-f5e2-4595-b6ae-6e95adc83cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:12.988402 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3836f0-f5e2-4595-b6ae-6e95adc83cec None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:12.988695 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3836f0-f5e2-4595-b6ae-6e95adc83cec None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:12.989242 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3836f0-f5e2-4595-b6ae-6e95adc83cec None admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:12.990040 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3836f0-f5e2-4595-b6ae-6e95adc83cec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:12.990381 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3836f0-f5e2-4595-b6ae-6e95adc83cec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:12.993020 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3836f0-f5e2-4595-b6ae-6e95adc83cec None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:12.995773 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:13.009547 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 332/676] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jun 21 18:06:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:13.010517 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3836f0-f5e2-4595-b6ae-6e95adc83cec None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:13.035539 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92c604c-1ddb-45f8-bc08-aff06dc71a61 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:13.074263 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18b7b70-58d1-476a-a4bf-876243b570b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:13.074935 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18b7b70-58d1-476a-a4bf-876243b570b0 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:13.075219 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18b7b70-58d1-476a-a4bf-876243b570b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:13.075495 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18b7b70-58d1-476a-a4bf-876243b570b0 None admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:13.076404 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18b7b70-58d1-476a-a4bf-876243b570b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:13.076734 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18b7b70-58d1-476a-a4bf-876243b570b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:13.078161 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18b7b70-58d1-476a-a4bf-876243b570b0 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:13.089165 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:13.093138 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 345/677] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:06:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:13.094068 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18b7b70-58d1-476a-a4bf-876243b570b0 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:13.140434 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3836f0-f5e2-4595-b6ae-6e95adc83cec None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:13.178253 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4b08d-1dec-40e7-aa27-3c297dd0182c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:13.179230 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4b08d-1dec-40e7-aa27-3c297dd0182c None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:13.179609 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4b08d-1dec-40e7-aa27-3c297dd0182c None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:13.179978 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4b08d-1dec-40e7-aa27-3c297dd0182c None admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:13.181538 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c4b08d-1dec-40e7-aa27-3c297dd0182c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:13.182561 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c4b08d-1dec-40e7-aa27-3c297dd0182c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:13.184060 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c4b08d-1dec-40e7-aa27-3c297dd0182c None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:13.203382 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 333/678] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:06:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:13.203382 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4b08d-1dec-40e7-aa27-3c297dd0182c None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:13.228203 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18b7b70-58d1-476a-a4bf-876243b570b0 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:13.270980 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:13.273514 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9779a9-7d3a-40e0-afb3-58f04bd1852f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:13.274348 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9779a9-7d3a-40e0-afb3-58f04bd1852f None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:13.274986 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9779a9-7d3a-40e0-afb3-58f04bd1852f None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:13.275412 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9779a9-7d3a-40e0-afb3-58f04bd1852f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:13.276974 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9779a9-7d3a-40e0-afb3-58f04bd1852f None admin] RBAC: Adding query filter params (role.id=acf4e26e0ceb41ddad002dd4edf6373c, scope.project.id=2a4699d1ec70456da5f8e0aa4b1439b7, user.id=9672794bc1ca4dcd8259d51d9290384b) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:13.277441 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9779a9-7d3a-40e0-afb3-58f04bd1852f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:13.279317 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9779a9-7d3a-40e0-afb3-58f04bd1852f None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:13.287864 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 346/679] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jun 21 18:06:13 2022] GET /identity/v3/role_assignments?user.id=9672794bc1ca4dcd8259d51d9290384b&scope.project.id=2a4699d1ec70456da5f8e0aa4b1439b7&role.id=acf4e26e0ceb41ddad002dd4edf6373c => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:13.289180 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9779a9-7d3a-40e0-afb3-58f04bd1852f None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:13.324511 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4b08d-1dec-40e7-aa27-3c297dd0182c None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:13.369088 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25a77d3-6b1b-43a3-b933-2a2fc7718abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:13.370754 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25a77d3-6b1b-43a3-b933-2a2fc7718abf None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:13.371119 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25a77d3-6b1b-43a3-b933-2a2fc7718abf None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:13.371536 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25a77d3-6b1b-43a3-b933-2a2fc7718abf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:13.379931 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25a77d3-6b1b-43a3-b933-2a2fc7718abf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:13.381151 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:13.383093 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25a77d3-6b1b-43a3-b933-2a2fc7718abf None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:13.401088 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-a25a77d3-6b1b-43a3-b933-2a2fc7718abf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:06:13.401088 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:13.401088 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:13.401088 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:13.401088 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:13.401088 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:13.401088 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:13.401088 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:13.401088 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:13.401088 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:13.401088 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:13.401088 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:13.401088 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:06:13.401088 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:06:13.401088 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:06:13.402287 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:06:13.402287 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:13.402287 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:13.402287 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:13.402287 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:13.402287 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:13.402287 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:13.402287 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:13.402287 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:13.402287 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:13.402287 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:13.402287 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:13.402287 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:13.402287 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:13.402287 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:13.402287 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:13.402287 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:13.403364 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:06:13.403364 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:06:13.403364 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:06:13.403364 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:06:13.403364 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:06:13.403364 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:06:13.403364 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:06:13.403364 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:13.404784 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 334/680] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jun 21 18:06:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:13.406319 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2093af-f8f5-4845-bd5a-98c4bdae3484 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:13.407240 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2093af-f8f5-4845-bd5a-98c4bdae3484 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:13.407240 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2093af-f8f5-4845-bd5a-98c4bdae3484 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:13.408807 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 335/681] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:06:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:06:13.415725 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d65c1a-3e1d-422d-865a-55f4eca808ed None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:13.416303 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d65c1a-3e1d-422d-865a-55f4eca808ed None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:13.417319 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d65c1a-3e1d-422d-865a-55f4eca808ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:13.430469 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9779a9-7d3a-40e0-afb3-58f04bd1852f None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:13.440768 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-10d65c1a-3e1d-422d-865a-55f4eca808ed None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:06:13.444202 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-10d65c1a-3e1d-422d-865a-55f4eca808ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:06:13.480567 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9c9cb1-232b-442b-9066-191753070a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:13.481720 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9c9cb1-232b-442b-9066-191753070a46 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:13.482206 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9c9cb1-232b-442b-9066-191753070a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:13.482668 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9c9cb1-232b-442b-9066-191753070a46 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:13.492350 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9c9cb1-232b-442b-9066-191753070a46 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:13.493195 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9c9cb1-232b-442b-9066-191753070a46 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:13.513715 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-fb9c9cb1-232b-442b-9066-191753070a46 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 21 18:06:13.513715 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:13.513715 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:13.513715 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:13.513715 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:13.513715 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:13.513715 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:13.513715 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:13.513715 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:13.513715 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:13.513715 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:13.513715 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:13.513715 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:06:13.513715 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:06:13.513715 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:06:13.513715 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:06:13.514995 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:13.514995 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:13.514995 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:06:13.514995 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:06:13.514995 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:06:13.514995 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:06:13.514995 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:13.514995 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:13.514995 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:13.514995 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:13.514995 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:13.514995 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:13.514995 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:13.514995 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:13.514995 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:13.514995 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:13.514995 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:13.514995 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:13.515976 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:13.515976 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:13.515976 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:06:13.515976 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:06:13.515976 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:06:13.515976 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:06:13.515976 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:06:13.515976 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:06:13.515976 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 21 18:06:13.515976 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:13.515976 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 347/682] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jun 21 18:06:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:13.516641 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9c9cb1-232b-442b-9066-191753070a46 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:13.533139 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 336/683] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:06:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:06:13.534062 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d65c1a-3e1d-422d-865a-55f4eca808ed None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:13.587416 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:13.651858 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d65c1a-3e1d-422d-865a-55f4eca808ed None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:13.656092 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9c9cb1-232b-442b-9066-191753070a46 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:13.690105 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bf83a2-07e4-4df3-a34b-574288665ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:13.690806 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bf83a2-07e4-4df3-a34b-574288665ee0 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:13.691092 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bf83a2-07e4-4df3-a34b-574288665ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:13.691266 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:13.691367 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bf83a2-07e4-4df3-a34b-574288665ee0 None admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:13.692285 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bf83a2-07e4-4df3-a34b-574288665ee0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:13.692628 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bf83a2-07e4-4df3-a34b-574288665ee0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:13.694171 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bf83a2-07e4-4df3-a34b-574288665ee0 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:13.708827 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae50940-ba41-41c7-8749-3e0bbcb7758b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:13.709979 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae50940-ba41-41c7-8749-3e0bbcb7758b None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:13.710478 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae50940-ba41-41c7-8749-3e0bbcb7758b None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:13.710974 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae50940-ba41-41c7-8749-3e0bbcb7758b None admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:13.711798 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae50940-ba41-41c7-8749-3e0bbcb7758b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:13.712809 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae50940-ba41-41c7-8749-3e0bbcb7758b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:13.715662 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae50940-ba41-41c7-8749-3e0bbcb7758b None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:13.718646 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 337/684] 10.222.0.23 () {56 vars in 1135 bytes} [Tue Jun 21 18:06:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:13.719578 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bf83a2-07e4-4df3-a34b-574288665ee0 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:13.731960 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 348/685] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jun 21 18:06:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:13.732876 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae50940-ba41-41c7-8749-3e0bbcb7758b None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:13.854383 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae50940-ba41-41c7-8749-3e0bbcb7758b None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:13.856444 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bf83a2-07e4-4df3-a34b-574288665ee0 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:13.858224 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:13.896658 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5f9b4a-f8a4-4cba-8262-2ff203a8ebaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:13.897623 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f9b4a-f8a4-4cba-8262-2ff203a8ebaa None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:13.898136 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f9b4a-f8a4-4cba-8262-2ff203a8ebaa None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:13.898641 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f9b4a-f8a4-4cba-8262-2ff203a8ebaa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:13.900974 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c44987b-f974-477c-aeaa-3e609abd0581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:13.901777 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44987b-f974-477c-aeaa-3e609abd0581 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:13.902063 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44987b-f974-477c-aeaa-3e609abd0581 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:13.902341 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44987b-f974-477c-aeaa-3e609abd0581 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:13.919649 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c44987b-f974-477c-aeaa-3e609abd0581 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:06:13.930601 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-8c44987b-f974-477c-aeaa-3e609abd0581 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:06:13.930601 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:13.930601 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:13.930601 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:13.930601 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:13.930601 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:13.930601 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:13.930601 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:13.930601 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:13.930601 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:13.930601 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:13.930601 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:13.930601 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:06:13.930601 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:06:13.930601 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:06:13.930601 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:06:13.932470 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:06:13.932470 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:06:13.932470 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:06:13.932470 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:06:13.932470 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:13.932470 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:13.932470 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:13.932470 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:13.932470 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:13.932470 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:13.932470 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:13.932470 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:13.932470 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:13.932470 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:13.932470 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:13.932470 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:13.933367 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:13.933367 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:13.933367 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:13.933367 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:13.933367 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:06:13.933367 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:06:13.933367 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:06:13.933367 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:06:13.933367 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:06:13.933367 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:06:13.933367 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:06:13.933367 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:13.936474 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 338/686] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jun 21 18:06:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:13.937468 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c44987b-f974-477c-aeaa-3e609abd0581 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:13.976116 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:13.998535 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5f9b4a-f8a4-4cba-8262-2ff203a8ebaa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:14.000619 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5f9b4a-f8a4-4cba-8262-2ff203a8ebaa None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:14.054653 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c44987b-f974-477c-aeaa-3e609abd0581 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:14.102138 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d87f844-6c8c-43cc-8a34-e2d0f2dfac6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:14.103996 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87f844-6c8c-43cc-8a34-e2d0f2dfac6d None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:14.104354 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87f844-6c8c-43cc-8a34-e2d0f2dfac6d None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:14.104735 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87f844-6c8c-43cc-8a34-e2d0f2dfac6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:14.105693 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d87f844-6c8c-43cc-8a34-e2d0f2dfac6d None admin] RBAC: Adding query filter params (role.id=acf4e26e0ceb41ddad002dd4edf6373c, scope.project.id=2a4699d1ec70456da5f8e0aa4b1439b7, user.id=59c7f07e58ba4db9a7d402d8f3da1e89) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:14.106126 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d87f844-6c8c-43cc-8a34-e2d0f2dfac6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:14.108072 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d87f844-6c8c-43cc-8a34-e2d0f2dfac6d None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:14.124421 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 339/687] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jun 21 18:06:13 2022] GET /identity/v3/role_assignments?user.id=59c7f07e58ba4db9a7d402d8f3da1e89&scope.project.id=2a4699d1ec70456da5f8e0aa4b1439b7&role.id=acf4e26e0ceb41ddad002dd4edf6373c => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:14.124421 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d87f844-6c8c-43cc-8a34-e2d0f2dfac6d None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:14.143513 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 349/688] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:06:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:06:14.149099 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5f9b4a-f8a4-4cba-8262-2ff203a8ebaa None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:14.153831 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:14.249930 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d87f844-6c8c-43cc-8a34-e2d0f2dfac6d None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:14.294745 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25f6f86-f157-4f47-a5a5-f944d5bc9c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:14.295446 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f6f86-f157-4f47-a5a5-f944d5bc9c7a None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:14.295811 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f6f86-f157-4f47-a5a5-f944d5bc9c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:14.296093 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f6f86-f157-4f47-a5a5-f944d5bc9c7a None admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:14.296882 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25f6f86-f157-4f47-a5a5-f944d5bc9c7a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:14.297191 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25f6f86-f157-4f47-a5a5-f944d5bc9c7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:14.299189 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25f6f86-f157-4f47-a5a5-f944d5bc9c7a None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:14.309853 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 340/689] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:06:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:14.315770 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25f6f86-f157-4f47-a5a5-f944d5bc9c7a None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:14.362433 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:14.381962 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5f9b4a-f8a4-4cba-8262-2ff203a8ebaa None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:14.419173 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432cf346-90e1-4bdc-84f9-1c0fd743881e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:14.420115 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432cf346-90e1-4bdc-84f9-1c0fd743881e None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:14.420238 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432cf346-90e1-4bdc-84f9-1c0fd743881e None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:14.421240 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432cf346-90e1-4bdc-84f9-1c0fd743881e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:14.431730 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432cf346-90e1-4bdc-84f9-1c0fd743881e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:14.432189 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:14.433257 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432cf346-90e1-4bdc-84f9-1c0fd743881e None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:14.445361 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25f6f86-f157-4f47-a5a5-f944d5bc9c7a None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:14.451406 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-432cf346-90e1-4bdc-84f9-1c0fd743881e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:14.451406 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:14.451406 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:14.451406 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:14.451406 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:14.451406 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:14.451406 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:14.451406 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:14.451406 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:14.451406 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:14.451406 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:14.451406 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:14.451406 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:06:14.451406 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:06:14.451406 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:06:14.452354 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:06:14.452354 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:14.452354 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:14.452354 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:14.452354 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:14.452354 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:14.452354 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:14.452354 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:14.452354 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:14.452354 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:14.452354 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:14.452354 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:14.452354 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:14.452354 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:14.452354 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:14.452354 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:14.452354 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:14.453297 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:06:14.453297 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:06:14.453297 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:14.453297 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:14.453297 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 350/690] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:06:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:14.459317 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432cf346-90e1-4bdc-84f9-1c0fd743881e None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:14.496877 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d31ef3c-48c7-4bd3-944d-634c506a9094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:14.498074 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31ef3c-48c7-4bd3-944d-634c506a9094 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:14.498372 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31ef3c-48c7-4bd3-944d-634c506a9094 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:14.498654 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31ef3c-48c7-4bd3-944d-634c506a9094 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:14.512374 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d31ef3c-48c7-4bd3-944d-634c506a9094 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:14.513894 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d31ef3c-48c7-4bd3-944d-634c506a9094 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:14.533743 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-8d31ef3c-48c7-4bd3-944d-634c506a9094 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:14.533743 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:14.533743 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:14.533743 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:14.533743 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:14.533743 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:14.533743 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:14.533743 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:14.533743 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:14.533743 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:14.533743 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:14.533743 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:14.533743 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:06:14.533743 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:06:14.533743 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:06:14.534715 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:06:14.534715 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:14.534715 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:14.534715 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:14.534715 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:14.534715 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:14.534715 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:14.534715 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:14.534715 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:14.534715 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:14.534715 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:14.534715 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:14.534715 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:14.534715 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:14.534715 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:14.534715 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:14.534715 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:14.536523 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:06:14.536523 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:06:14.536523 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:14.536523 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:14.536523 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 341/691] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:06:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:14.540897 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d31ef3c-48c7-4bd3-944d-634c506a9094 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:14.592000 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432cf346-90e1-4bdc-84f9-1c0fd743881e None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:14.628755 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcb8576-4739-44e3-b1ca-6dce1083cb77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:14.629188 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb8576-4739-44e3-b1ca-6dce1083cb77 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:14.629471 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb8576-4739-44e3-b1ca-6dce1083cb77 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:14.629747 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb8576-4739-44e3-b1ca-6dce1083cb77 None admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:14.630481 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcb8576-4739-44e3-b1ca-6dce1083cb77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:14.630788 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcb8576-4739-44e3-b1ca-6dce1083cb77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:14.632273 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcb8576-4739-44e3-b1ca-6dce1083cb77 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:14.635596 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:14.657198 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 351/692] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:06:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:06:14.664264 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcb8576-4739-44e3-b1ca-6dce1083cb77 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:14.694607 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d31ef3c-48c7-4bd3-944d-634c506a9094 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:14.779614 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcb8576-4739-44e3-b1ca-6dce1083cb77 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:14.781385 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8979e48a-bc07-429e-bbeb-7e0bd1d15641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:14.782165 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8979e48a-bc07-429e-bbeb-7e0bd1d15641 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:14.782553 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8979e48a-bc07-429e-bbeb-7e0bd1d15641 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:14.782955 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8979e48a-bc07-429e-bbeb-7e0bd1d15641 None admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:14.783789 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8979e48a-bc07-429e-bbeb-7e0bd1d15641 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:14.784206 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8979e48a-bc07-429e-bbeb-7e0bd1d15641 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:14.785726 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8979e48a-bc07-429e-bbeb-7e0bd1d15641 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:14.795462 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:14.800108 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 342/693] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:06:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:06:14.801656 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fad13a-cb05-4fb6-95bb-8501d6e4632d None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:14.802186 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fad13a-cb05-4fb6-95bb-8501d6e4632d None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:14.802670 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fad13a-cb05-4fb6-95bb-8501d6e4632d None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:14.808436 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 343/694] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:06:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:06:14.809572 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fad13a-cb05-4fb6-95bb-8501d6e4632d None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:14.828310 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf3541a-158e-4952-bfd6-ab7fa33258fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:14.829321 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf3541a-158e-4952-bfd6-ab7fa33258fe None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:14.829662 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf3541a-158e-4952-bfd6-ab7fa33258fe None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:14.829939 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf3541a-158e-4952-bfd6-ab7fa33258fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:14.839950 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf3541a-158e-4952-bfd6-ab7fa33258fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:14.842050 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf3541a-158e-4952-bfd6-ab7fa33258fe None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:14.859974 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-9bf3541a-158e-4952-bfd6-ab7fa33258fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:06:14.859974 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:14.859974 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:14.859974 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:14.859974 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:14.859974 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:14.859974 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:14.859974 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:14.859974 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:14.859974 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:14.859974 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:14.859974 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:14.859974 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:06:14.859974 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:06:14.859974 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:06:14.861002 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:06:14.861002 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:14.861002 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:14.861002 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:14.861002 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:14.861002 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:14.861002 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:14.861002 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:14.861002 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:14.861002 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:14.861002 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:14.861002 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:14.861002 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:14.861002 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:14.861002 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:14.861002 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:14.861002 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:14.861968 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:06:14.861968 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:06:14.861968 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:06:14.861968 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:06:14.861968 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:06:14.861968 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:06:14.861968 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:06:14.861968 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:14.861968 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 352/695] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jun 21 18:06:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:14.863007 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4ce24a-8481-445d-8ac6-110afb60937e None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:14.863253 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4ce24a-8481-445d-8ac6-110afb60937e None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:14.863536 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4ce24a-8481-445d-8ac6-110afb60937e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:14.884775 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-2c4ce24a-8481-445d-8ac6-110afb60937e None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:06:14.886085 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-2c4ce24a-8481-445d-8ac6-110afb60937e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:06:14.935666 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fad13a-cb05-4fb6-95bb-8501d6e4632d None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:14.954325 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:14.979775 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 353/696] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:06:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:06:14.980858 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4ce24a-8481-445d-8ac6-110afb60937e None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:14.983175 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3129a9-4781-48cf-a96f-91b13c73dfc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:14.984231 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3129a9-4781-48cf-a96f-91b13c73dfc6 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:14.984777 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3129a9-4781-48cf-a96f-91b13c73dfc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:14.985304 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3129a9-4781-48cf-a96f-91b13c73dfc6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:14.995793 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3129a9-4781-48cf-a96f-91b13c73dfc6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:14.997626 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3129a9-4781-48cf-a96f-91b13c73dfc6 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:15.017663 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-cb3129a9-4781-48cf-a96f-91b13c73dfc6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 21 18:06:15.017663 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:15.017663 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:15.017663 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:15.017663 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:15.017663 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:15.017663 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:15.017663 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:15.017663 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:15.017663 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:15.017663 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:15.017663 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:15.017663 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:06:15.017663 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:06:15.017663 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:06:15.017663 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:06:15.018701 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:15.018701 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:15.018701 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:06:15.018701 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:06:15.018701 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:06:15.018701 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:06:15.018701 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:15.018701 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:15.018701 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:15.018701 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:15.018701 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:15.018701 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:15.018701 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:15.018701 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:15.018701 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:15.018701 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:15.018701 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:15.018701 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:15.019706 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:15.019706 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:15.019706 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:06:15.019706 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:06:15.019706 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:06:15.019706 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:06:15.019706 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:06:15.019706 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:06:15.019706 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 21 18:06:15.019706 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:15.022660 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 344/697] 10.222.0.23 () {56 vars in 1093 bytes} [Tue Jun 21 18:06:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:15.028071 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3129a9-4781-48cf-a96f-91b13c73dfc6 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:15.096532 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:15.135886 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4ce24a-8481-445d-8ac6-110afb60937e None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:15.160927 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3129a9-4781-48cf-a96f-91b13c73dfc6 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:15.185115 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1b361e-e30d-4325-8577-252f285a4c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:15.186222 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b361e-e30d-4325-8577-252f285a4c67 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:15.186992 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b361e-e30d-4325-8577-252f285a4c67 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:15.187422 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b361e-e30d-4325-8577-252f285a4c67 None admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:15.188630 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1b361e-e30d-4325-8577-252f285a4c67 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:15.189239 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1b361e-e30d-4325-8577-252f285a4c67 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:15.191421 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1b361e-e30d-4325-8577-252f285a4c67 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:15.199194 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1058cc6b-de8a-4c4f-92a8-7e9b6c26de8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:15.200201 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1058cc6b-de8a-4c4f-92a8-7e9b6c26de8d None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:15.200620 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1058cc6b-de8a-4c4f-92a8-7e9b6c26de8d None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:15.201008 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1058cc6b-de8a-4c4f-92a8-7e9b6c26de8d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:15.202157 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1058cc6b-de8a-4c4f-92a8-7e9b6c26de8d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:15.203475 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1058cc6b-de8a-4c4f-92a8-7e9b6c26de8d None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:15.208242 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 354/698] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jun 21 18:06:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:15.209649 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1b361e-e30d-4325-8577-252f285a4c67 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:15.220536 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-1058cc6b-de8a-4c4f-92a8-7e9b6c26de8d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 21 18:06:15.220536 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:15.220536 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:06:15.220536 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:06:15.220536 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:06:15.220536 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:06:15.220536 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:06:15.220536 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:06:15.220536 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 21 18:06:15.220536 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:15.220536 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:06:15.220536 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:15.220536 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:15.220536 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:15.220536 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:15.220536 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:15.221723 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:15.221723 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:15.221723 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:15.221723 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:15.221723 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:15.221723 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:15.221723 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:15.221723 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:06:15.221723 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:06:15.221723 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:06:15.221723 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:06:15.221723 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:15.221723 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:15.221723 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:15.221723 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:15.221723 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:15.222777 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:15.222777 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:15.222777 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:15.222777 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:15.222777 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:15.222777 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:15.222777 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:15.222777 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:15.222777 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:15.222777 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:15.222777 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:15.222777 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:06:15.222777 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:06:15.222777 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 21 18:06:15.222777 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:15.230809 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 345/699] 10.222.0.23 () {56 vars in 1129 bytes} [Tue Jun 21 18:06:15 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:15.231867 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1058cc6b-de8a-4c4f-92a8-7e9b6c26de8d None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:15.269377 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:15.332485 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1b361e-e30d-4325-8577-252f285a4c67 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:15.370288 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b96f434-6f52-4eb2-96d7-eeaae0b2ca8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:15.371127 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b96f434-6f52-4eb2-96d7-eeaae0b2ca8e None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:15.371518 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b96f434-6f52-4eb2-96d7-eeaae0b2ca8e None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:15.371906 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b96f434-6f52-4eb2-96d7-eeaae0b2ca8e None admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:15.372910 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b96f434-6f52-4eb2-96d7-eeaae0b2ca8e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:15.373332 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b96f434-6f52-4eb2-96d7-eeaae0b2ca8e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:15.373960 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1058cc6b-de8a-4c4f-92a8-7e9b6c26de8d None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:15.374999 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b96f434-6f52-4eb2-96d7-eeaae0b2ca8e None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:15.400808 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 355/700] 10.222.0.23 () {56 vars in 1133 bytes} [Tue Jun 21 18:06:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:15.400808 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b96f434-6f52-4eb2-96d7-eeaae0b2ca8e None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:15.410662 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:15.425007 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26509c77-dc51-4729-9456-5b4c9e4f3e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:15.426009 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26509c77-dc51-4729-9456-5b4c9e4f3e13 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:15.426437 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26509c77-dc51-4729-9456-5b4c9e4f3e13 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:15.427090 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26509c77-dc51-4729-9456-5b4c9e4f3e13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:15.428564 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26509c77-dc51-4729-9456-5b4c9e4f3e13 None admin] RBAC: Adding query filter params (role.id=acf4e26e0ceb41ddad002dd4edf6373c, scope.project.id=2a4699d1ec70456da5f8e0aa4b1439b7, user.id=0e0d25565f1543d9a7577288fc09119f) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:15.429275 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26509c77-dc51-4729-9456-5b4c9e4f3e13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:15.432153 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26509c77-dc51-4729-9456-5b4c9e4f3e13 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:15.441090 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 346/701] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jun 21 18:06:15 2022] GET /identity/v3/role_assignments?user.id=0e0d25565f1543d9a7577288fc09119f&scope.project.id=2a4699d1ec70456da5f8e0aa4b1439b7&role.id=acf4e26e0ceb41ddad002dd4edf6373c => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:15.442111 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26509c77-dc51-4729-9456-5b4c9e4f3e13 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:15.578659 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26509c77-dc51-4729-9456-5b4c9e4f3e13 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:15.620282 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7975fc3c-9fda-4bf3-876b-ef4c579a4f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:15.622034 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7975fc3c-9fda-4bf3-876b-ef4c579a4f4e None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:15.622488 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7975fc3c-9fda-4bf3-876b-ef4c579a4f4e None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:15.622961 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7975fc3c-9fda-4bf3-876b-ef4c579a4f4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:15.666472 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b96f434-6f52-4eb2-96d7-eeaae0b2ca8e None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:15.669701 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:15.694136 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:15.705580 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7975fc3c-9fda-4bf3-876b-ef4c579a4f4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:15.707583 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7975fc3c-9fda-4bf3-876b-ef4c579a4f4e None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:15.712995 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f5c55b-8cee-40e5-b3ad-13d5d36f5668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:15.713710 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f5c55b-8cee-40e5-b3ad-13d5d36f5668 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:15.713998 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f5c55b-8cee-40e5-b3ad-13d5d36f5668 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:15.714281 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f5c55b-8cee-40e5-b3ad-13d5d36f5668 None admin] PATH_INFO: `/v3/services` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:15.715007 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f5c55b-8cee-40e5-b3ad-13d5d36f5668 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:15.715318 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f5c55b-8cee-40e5-b3ad-13d5d36f5668 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:15.716735 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f5c55b-8cee-40e5-b3ad-13d5d36f5668 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:15.722195 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 356/702] 10.222.0.23 () {56 vars in 1124 bytes} [Tue Jun 21 18:06:15 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:15.729517 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd088277-61df-40ed-b603-63a18388aeb9 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:15.729802 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd088277-61df-40ed-b603-63a18388aeb9 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:15.730108 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd088277-61df-40ed-b603-63a18388aeb9 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:15.731515 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 357/703] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:06:15 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:06:15.732226 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd088277-61df-40ed-b603-63a18388aeb9 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:15.841202 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 347/704] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:06:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:06:15.846007 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cecf438-ca96-46f8-864b-1fe14cb14406 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:15.846007 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cecf438-ca96-46f8-864b-1fe14cb14406 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:15.846007 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cecf438-ca96-46f8-864b-1fe14cb14406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:15.863025 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-0cecf438-ca96-46f8-864b-1fe14cb14406 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:06:15.864528 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-0cecf438-ca96-46f8-864b-1fe14cb14406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:06:15.867332 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd088277-61df-40ed-b603-63a18388aeb9 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:15.906669 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1878a188-5d35-470e-8b77-6a3a6edeb39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:15.907343 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878a188-5d35-470e-8b77-6a3a6edeb39c None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:15.907625 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878a188-5d35-470e-8b77-6a3a6edeb39c None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:15.907902 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878a188-5d35-470e-8b77-6a3a6edeb39c None admin] PATH_INFO: `/v3/services` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:15.908765 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1878a188-5d35-470e-8b77-6a3a6edeb39c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:15.908952 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1878a188-5d35-470e-8b77-6a3a6edeb39c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:15.910737 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1878a188-5d35-470e-8b77-6a3a6edeb39c None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:15.916827 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 358/705] 10.222.0.23 () {56 vars in 1124 bytes} [Tue Jun 21 18:06:15 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:15.917659 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1878a188-5d35-470e-8b77-6a3a6edeb39c None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:15.948973 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 348/706] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:06:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:06:15.951685 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:15.975327 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cecf438-ca96-46f8-864b-1fe14cb14406 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:16.021209 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:16.135024 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cecf438-ca96-46f8-864b-1fe14cb14406 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:16.147878 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1878a188-5d35-470e-8b77-6a3a6edeb39c None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:16.174364 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdf9ff3-65dd-485f-8795-a010a2c21dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:16.175520 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdf9ff3-65dd-485f-8795-a010a2c21dde None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:16.176019 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdf9ff3-65dd-485f-8795-a010a2c21dde None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:16.176557 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdf9ff3-65dd-485f-8795-a010a2c21dde None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:16.188574 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdf9ff3-65dd-485f-8795-a010a2c21dde None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:16.190403 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdf9ff3-65dd-485f-8795-a010a2c21dde None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:16.209844 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-ebdf9ff3-65dd-485f-8795-a010a2c21dde None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:16.209844 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:16.209844 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:16.209844 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:16.209844 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:16.209844 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:16.209844 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:16.209844 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:16.209844 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:16.209844 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:16.209844 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:16.209844 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:16.209844 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:06:16.209844 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:06:16.209844 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:06:16.211028 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:06:16.211028 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:16.211028 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:16.211028 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:16.211028 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:16.211028 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:16.211028 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:16.211028 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:16.211028 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:16.211028 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:16.211028 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:16.211028 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:16.211028 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:16.211028 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:16.211028 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:16.211028 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:16.211028 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:16.212178 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:06:16.212178 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:06:16.212178 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:16.212178 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:16.214726 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 349/707] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:06:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:16.220920 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdf9ff3-65dd-485f-8795-a010a2c21dde None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:16.224461 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74363a75-c941-49c3-aa54-57692b246af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:16.225624 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74363a75-c941-49c3-aa54-57692b246af6 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:16.225840 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74363a75-c941-49c3-aa54-57692b246af6 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:16.226105 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74363a75-c941-49c3-aa54-57692b246af6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:16.244960 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74363a75-c941-49c3-aa54-57692b246af6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:16.246769 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74363a75-c941-49c3-aa54-57692b246af6 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:16.267206 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-74363a75-c941-49c3-aa54-57692b246af6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:16.267206 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:16.267206 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:16.267206 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:16.267206 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:16.267206 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:16.267206 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:16.267206 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:16.267206 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:16.267206 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:16.267206 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:16.267206 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:16.267206 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:06:16.267206 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:06:16.267206 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:06:16.268182 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:06:16.268182 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:16.268182 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:16.268182 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:16.268182 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:16.268182 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:16.268182 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:16.268182 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:16.268182 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:16.268182 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:16.268182 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:16.268182 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:16.268182 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:16.268182 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:16.268182 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:16.268182 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:16.268182 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:16.269145 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:06:16.269145 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:06:16.269145 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:16.269145 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:16.272299 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 359/708] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:06:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:16.278917 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74363a75-c941-49c3-aa54-57692b246af6 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:16.299952 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:16.335095 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdf9ff3-65dd-485f-8795-a010a2c21dde None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:16.375549 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe1d516-5df6-41b9-a579-7e89bd210e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:16.376233 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe1d516-5df6-41b9-a579-7e89bd210e57 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:16.376565 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe1d516-5df6-41b9-a579-7e89bd210e57 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:16.376847 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe1d516-5df6-41b9-a579-7e89bd210e57 None admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:16.377580 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe1d516-5df6-41b9-a579-7e89bd210e57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:16.377887 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe1d516-5df6-41b9-a579-7e89bd210e57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:16.380658 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe1d516-5df6-41b9-a579-7e89bd210e57 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:16.406414 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 350/709] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:06:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:16.412547 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe1d516-5df6-41b9-a579-7e89bd210e57 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:16.429361 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:16.447624 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74363a75-c941-49c3-aa54-57692b246af6 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:16.485739 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bca7818-4049-4cc3-bc24-1a5cfbb984d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:16.486882 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bca7818-4049-4cc3-bc24-1a5cfbb984d0 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:16.487381 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bca7818-4049-4cc3-bc24-1a5cfbb984d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:16.487881 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bca7818-4049-4cc3-bc24-1a5cfbb984d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:16.488970 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bca7818-4049-4cc3-bc24-1a5cfbb984d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:16.489526 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bca7818-4049-4cc3-bc24-1a5cfbb984d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:16.491147 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bca7818-4049-4cc3-bc24-1a5cfbb984d0 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:16.512920 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 360/710] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:06:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:06:16.519093 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bca7818-4049-4cc3-bc24-1a5cfbb984d0 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:16.522473 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe1d516-5df6-41b9-a579-7e89bd210e57 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:16.554238 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:16.565354 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1988dcab-2d30-4ac4-be2f-570128925f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:16.565354 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1988dcab-2d30-4ac4-be2f-570128925f81 None admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:16.565354 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1988dcab-2d30-4ac4-be2f-570128925f81 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:16.565354 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1988dcab-2d30-4ac4-be2f-570128925f81 None admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:16.567959 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1988dcab-2d30-4ac4-be2f-570128925f81 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:16.567959 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1988dcab-2d30-4ac4-be2f-570128925f81 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:16.657366 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bca7818-4049-4cc3-bc24-1a5cfbb984d0 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:16.689232 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 351/711] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Jun 21 18:06:16 2022] POST /identity/v3/users => generated 254 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:06:16.689232 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aad6bb-0c1b-441b-9e60-c7331ee8811d None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:16.689232 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aad6bb-0c1b-441b-9e60-c7331ee8811d None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:16.689232 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aad6bb-0c1b-441b-9e60-c7331ee8811d None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:16.689232 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 352/712] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:06:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:06:16.695833 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736baba-d019-4d92-a4a2-aa427260f7af None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:16.696198 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736baba-d019-4d92-a4a2-aa427260f7af None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:16.697564 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736baba-d019-4d92-a4a2-aa427260f7af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:16.730071 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439cebb2-7b26-445e-82e4-a7d1bd4483ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:16.731339 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439cebb2-7b26-445e-82e4-a7d1bd4483ef None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:16.731909 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439cebb2-7b26-445e-82e4-a7d1bd4483ef None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:16.732495 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439cebb2-7b26-445e-82e4-a7d1bd4483ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:16.738364 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-4736baba-d019-4d92-a4a2-aa427260f7af None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:06:16.741038 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-4736baba-d019-4d92-a4a2-aa427260f7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:06:16.748852 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439cebb2-7b26-445e-82e4-a7d1bd4483ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:16.751637 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439cebb2-7b26-445e-82e4-a7d1bd4483ef None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:16.782009 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-439cebb2-7b26-445e-82e4-a7d1bd4483ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:06:16.782009 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:16.782009 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:16.782009 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:16.782009 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:16.782009 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:16.782009 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:16.782009 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:16.782009 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:16.782009 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:16.782009 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:16.782009 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:16.782009 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:06:16.782009 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:06:16.782009 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:06:16.783036 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:06:16.783036 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:16.783036 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:16.783036 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:16.783036 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:16.783036 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:16.783036 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:16.783036 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:16.783036 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:16.783036 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:16.783036 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:16.783036 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:16.783036 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:16.783036 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:16.783036 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:16.783036 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:16.783036 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:16.784064 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:06:16.784064 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:06:16.784064 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:06:16.784064 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:06:16.784064 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:06:16.784064 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:06:16.784064 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:06:16.784064 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:16.793306 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 361/713] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jun 21 18:06:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:06:16.801531 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439cebb2-7b26-445e-82e4-a7d1bd4483ef None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:16.806368 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:16.848605 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 353/714] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:06:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:06:16.908571 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4736baba-d019-4d92-a4a2-aa427260f7af None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:16.955951 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439cebb2-7b26-445e-82e4-a7d1bd4483ef None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:16.993716 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097d0f6f-4400-49da-8964-99219c927f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:16.994758 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d0f6f-4400-49da-8964-99219c927f05 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:16.995256 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d0f6f-4400-49da-8964-99219c927f05 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:16.995762 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d0f6f-4400-49da-8964-99219c927f05 None admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:16.996793 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097d0f6f-4400-49da-8964-99219c927f05 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:16.997312 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097d0f6f-4400-49da-8964-99219c927f05 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:16.999012 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097d0f6f-4400-49da-8964-99219c927f05 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:17.011791 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 362/715] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jun 21 18:06:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:17.021065 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097d0f6f-4400-49da-8964-99219c927f05 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:17.038346 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4736baba-d019-4d92-a4a2-aa427260f7af None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:17.062731 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:17.092698 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d9cf6c-6fb5-4582-b067-4bd874e4d971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:17.093836 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d9cf6c-6fb5-4582-b067-4bd874e4d971 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:17.094434 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d9cf6c-6fb5-4582-b067-4bd874e4d971 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:17.094918 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d9cf6c-6fb5-4582-b067-4bd874e4d971 None admin] PATH_INFO: `/v3/services/network` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:17.096028 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d9cf6c-6fb5-4582-b067-4bd874e4d971 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:17.097656 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d9cf6c-6fb5-4582-b067-4bd874e4d971 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:17.111342 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:17.116568 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-18d9cf6c-6fb5-4582-b067-4bd874e4d971 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 21 18:06:17.116568 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:17.116568 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:06:17.116568 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:06:17.116568 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:06:17.116568 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:06:17.116568 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:06:17.116568 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:06:17.116568 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 21 18:06:17.116568 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:17.116568 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:06:17.116568 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:17.116568 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:17.116568 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:17.116568 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:17.116568 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:17.116568 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:17.117665 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:17.117665 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:17.117665 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:17.117665 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:17.117665 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:17.117665 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:17.117665 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:06:17.117665 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:06:17.117665 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:06:17.117665 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:06:17.117665 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:17.117665 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:17.117665 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:17.117665 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:17.117665 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:17.117665 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:17.119434 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:17.119434 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:17.119434 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:17.119434 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:17.119434 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:17.119434 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:17.119434 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:17.119434 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:17.119434 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:17.119434 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:17.119434 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:06:17.119434 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:06:17.119434 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 21 18:06:17.119434 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:17.124146 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 354/716] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jun 21 18:06:16 2022] GET /identity/v3/services/network => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:17.131506 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d9cf6c-6fb5-4582-b067-4bd874e4d971 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:17.184039 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097d0f6f-4400-49da-8964-99219c927f05 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:17.226985 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ebe8e4-f133-4260-9add-a39c54d09260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:17.228361 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ebe8e4-f133-4260-9add-a39c54d09260 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:17.228767 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ebe8e4-f133-4260-9add-a39c54d09260 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:17.229049 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ebe8e4-f133-4260-9add-a39c54d09260 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:17.229931 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ebe8e4-f133-4260-9add-a39c54d09260 None admin] RBAC: Adding query filter params (role.id=745ed9065a8143e384ebbd694b29635c, scope.project.id=2a4699d1ec70456da5f8e0aa4b1439b7, user.id=904255dd3c1f406eb10f6fc65f9cd098) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:17.230248 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ebe8e4-f133-4260-9add-a39c54d09260 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:17.231810 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ebe8e4-f133-4260-9add-a39c54d09260 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:17.239601 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 363/717] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jun 21 18:06:17 2022] GET /identity/v3/role_assignments?user.id=904255dd3c1f406eb10f6fc65f9cd098&scope.project.id=2a4699d1ec70456da5f8e0aa4b1439b7&role.id=745ed9065a8143e384ebbd694b29635c => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:06:17.275382 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d9cf6c-6fb5-4582-b067-4bd874e4d971 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:17.319846 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba0d350-9960-4f37-b32f-298e87d545fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:17.320755 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0d350-9960-4f37-b32f-298e87d545fd None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:17.321178 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0d350-9960-4f37-b32f-298e87d545fd None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:17.322205 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0d350-9960-4f37-b32f-298e87d545fd None admin] PATH_INFO: `/v3/services` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:17.323088 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba0d350-9960-4f37-b32f-298e87d545fd None admin] RBAC: Adding query filter params (name=network) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:17.323517 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba0d350-9960-4f37-b32f-298e87d545fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:17.325125 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba0d350-9960-4f37-b32f-298e87d545fd None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:17.331127 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 355/718] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Jun 21 18:06:17 2022] GET /identity/v3/services?name=network => generated 124 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:06:17.336471 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ebe8e4-f133-4260-9add-a39c54d09260 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:17.464551 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ebe8e4-f133-4260-9add-a39c54d09260 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:17.497958 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:17.507747 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a211de2-5e2d-47b8-bbbd-f45b92166d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:17.515502 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a211de2-5e2d-47b8-bbbd-f45b92166d4f None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:17.515502 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a211de2-5e2d-47b8-bbbd-f45b92166d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:17.515502 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a211de2-5e2d-47b8-bbbd-f45b92166d4f None admin] PATH_INFO: `/v3/services` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:17.515502 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a211de2-5e2d-47b8-bbbd-f45b92166d4f None admin] RBAC: Adding query filter params (type=network) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:17.515502 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a211de2-5e2d-47b8-bbbd-f45b92166d4f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:17.515502 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a211de2-5e2d-47b8-bbbd-f45b92166d4f None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:17.519136 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 364/719] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Jun 21 18:06:17 2022] GET /identity/v3/services?type=network => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:06:17.966040 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75cf5db-8164-4711-8f46-7ed6fd2a8791 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:17.966376 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75cf5db-8164-4711-8f46-7ed6fd2a8791 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:17.966615 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75cf5db-8164-4711-8f46-7ed6fd2a8791 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:17.968329 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 356/720] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:06:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:06:17.973833 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468cead6-075e-4820-8725-6ea26214fe76 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:17.974111 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468cead6-075e-4820-8725-6ea26214fe76 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:17.974390 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468cead6-075e-4820-8725-6ea26214fe76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:17.998412 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-468cead6-075e-4820-8725-6ea26214fe76 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:06:17.999597 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-468cead6-075e-4820-8725-6ea26214fe76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:06:18.098119 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 365/721] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:06:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:06:18.141463 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75cf5db-8164-4711-8f46-7ed6fd2a8791 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:18.210424 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:18.279626 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75cf5db-8164-4711-8f46-7ed6fd2a8791 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:18.365098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e3d6b2-08b3-45aa-b0c6-1190d60dca33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:18.366342 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e3d6b2-08b3-45aa-b0c6-1190d60dca33 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:18.366842 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e3d6b2-08b3-45aa-b0c6-1190d60dca33 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:18.367341 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e3d6b2-08b3-45aa-b0c6-1190d60dca33 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:18.391747 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e3d6b2-08b3-45aa-b0c6-1190d60dca33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:06:18.399910 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-99e3d6b2-08b3-45aa-b0c6-1190d60dca33 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:06:18.399910 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:18.399910 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:18.399910 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:18.399910 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:18.399910 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:18.399910 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:18.399910 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:18.399910 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:18.399910 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:18.399910 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:18.399910 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:18.399910 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:06:18.399910 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:06:18.399910 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:06:18.399910 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:06:18.400949 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:06:18.400949 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:06:18.400949 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:06:18.400949 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:06:18.400949 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:18.400949 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:18.400949 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:18.400949 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:18.400949 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:18.400949 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:18.400949 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:18.400949 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:18.400949 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:18.400949 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:18.400949 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:18.400949 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:18.401860 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:18.401860 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:18.401860 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:18.401860 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:18.401860 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:06:18.401860 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:06:18.401860 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:06:18.401860 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:06:18.401860 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:06:18.401860 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:06:18.401860 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:06:18.401860 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:18.411720 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 357/722] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jun 21 18:06:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:18.417393 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468cead6-075e-4820-8725-6ea26214fe76 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:18.466665 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e214840b-5e7d-4b39-b0ed-57473fc6d7f4 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:18.467341 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e214840b-5e7d-4b39-b0ed-57473fc6d7f4 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:18.467866 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e214840b-5e7d-4b39-b0ed-57473fc6d7f4 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:18.469760 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 358/723] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:06:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:06:18.475325 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509994a-6e8c-41b2-9e8e-19ffac4f8434 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:18.475896 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509994a-6e8c-41b2-9e8e-19ffac4f8434 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:18.476451 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509994a-6e8c-41b2-9e8e-19ffac4f8434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:18.521336 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-f509994a-6e8c-41b2-9e8e-19ffac4f8434 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:06:18.522943 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-f509994a-6e8c-41b2-9e8e-19ffac4f8434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:06:18.622504 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468cead6-075e-4820-8725-6ea26214fe76 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:18.680945 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:18.722496 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb13558e-a2b2-4b7f-83f2-3bb212f3ad41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:18.723204 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb13558e-a2b2-4b7f-83f2-3bb212f3ad41 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:18.728421 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb13558e-a2b2-4b7f-83f2-3bb212f3ad41 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:18.728421 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb13558e-a2b2-4b7f-83f2-3bb212f3ad41 None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:18.730103 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb13558e-a2b2-4b7f-83f2-3bb212f3ad41 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:18.730434 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb13558e-a2b2-4b7f-83f2-3bb212f3ad41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:18.731873 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb13558e-a2b2-4b7f-83f2-3bb212f3ad41 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:18.748903 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 359/724] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:06:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:06:18.772118 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f509994a-6e8c-41b2-9e8e-19ffac4f8434 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:18.786600 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 366/725] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:06:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:18.794723 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb13558e-a2b2-4b7f-83f2-3bb212f3ad41 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:18.828790 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:18.995570 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb13558e-a2b2-4b7f-83f2-3bb212f3ad41 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:19.065128 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044e68a8-12ba-4abe-a263-c8fc163b779b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:19.065887 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e68a8-12ba-4abe-a263-c8fc163b779b None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:19.066167 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e68a8-12ba-4abe-a263-c8fc163b779b None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:19.066436 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e68a8-12ba-4abe-a263-c8fc163b779b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:19.094567 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044e68a8-12ba-4abe-a263-c8fc163b779b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:19.096043 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044e68a8-12ba-4abe-a263-c8fc163b779b None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:19.129743 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-044e68a8-12ba-4abe-a263-c8fc163b779b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:19.129743 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:19.129743 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:19.129743 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:19.129743 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:19.129743 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:19.129743 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:19.129743 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:19.129743 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:19.129743 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:19.129743 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:19.129743 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:19.129743 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:06:19.129743 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:06:19.129743 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:06:19.130781 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:06:19.130781 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:19.130781 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:19.130781 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:19.130781 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:19.130781 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:19.130781 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:19.130781 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:19.130781 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:19.130781 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:19.130781 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:19.130781 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:19.130781 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:19.130781 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:19.130781 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:19.130781 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:19.130781 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:19.131691 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:06:19.131691 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:06:19.131691 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:19.131691 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:19.145086 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 367/726] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:06:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:19.148304 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044e68a8-12ba-4abe-a263-c8fc163b779b None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:19.195542 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f509994a-6e8c-41b2-9e8e-19ffac4f8434 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:19.254648 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:19.281633 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044e68a8-12ba-4abe-a263-c8fc163b779b None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:19.289208 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be62468-fc66-46fd-955e-1ef5577145e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:19.289970 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be62468-fc66-46fd-955e-1ef5577145e3 None admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:19.290340 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be62468-fc66-46fd-955e-1ef5577145e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:19.290613 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be62468-fc66-46fd-955e-1ef5577145e3 None admin] PATH_INFO: `/v3/services` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:19.291354 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be62468-fc66-46fd-955e-1ef5577145e3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:19.292754 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be62468-fc66-46fd-955e-1ef5577145e3 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:19.330476 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da5a522-943b-45d4-b6fe-660e9c8ce81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:19.331163 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da5a522-943b-45d4-b6fe-660e9c8ce81d None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:19.331453 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da5a522-943b-45d4-b6fe-660e9c8ce81d None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:19.331737 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da5a522-943b-45d4-b6fe-660e9c8ce81d None admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:19.332526 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da5a522-943b-45d4-b6fe-660e9c8ce81d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:19.332883 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da5a522-943b-45d4-b6fe-660e9c8ce81d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:19.334336 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da5a522-943b-45d4-b6fe-660e9c8ce81d None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:19.349312 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 368/727] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:06:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:19.355335 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da5a522-943b-45d4-b6fe-660e9c8ce81d None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:19.402689 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 360/728] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Jun 21 18:06:18 2022] POST /identity/v3/services => generated 251 bytes in 631 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:06:19.469935 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:19.476672 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da5a522-943b-45d4-b6fe-660e9c8ce81d None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:19.530240 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cd6df1-a604-4cce-b1f1-688073aaa60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:19.530968 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd6df1-a604-4cce-b1f1-688073aaa60f None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:19.531251 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd6df1-a604-4cce-b1f1-688073aaa60f None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:19.531535 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd6df1-a604-4cce-b1f1-688073aaa60f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:19.540165 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cd6df1-a604-4cce-b1f1-688073aaa60f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:19.541737 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cd6df1-a604-4cce-b1f1-688073aaa60f None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:19.557611 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-e8cd6df1-a604-4cce-b1f1-688073aaa60f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 21 18:06:19.557611 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:19.557611 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:19.557611 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:19.557611 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:19.557611 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:19.557611 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:19.557611 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:19.557611 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:19.557611 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:19.557611 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:19.557611 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:19.557611 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:06:19.557611 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:06:19.557611 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:06:19.557611 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:06:19.563051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:19.563051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:19.563051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:06:19.563051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:06:19.563051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:06:19.563051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:06:19.563051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:19.563051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:19.563051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:19.563051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:19.563051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:19.563051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:19.563051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:19.563051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:19.563051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:19.563051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:19.563051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:19.563051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:19.565565 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:19.565565 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:19.565565 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:06:19.565565 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:06:19.565565 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:06:19.565565 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:06:19.565565 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:06:19.565565 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:06:19.565565 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 21 18:06:19.565565 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:19.565565 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 369/729] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jun 21 18:06:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:19.567029 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be62468-fc66-46fd-955e-1ef5577145e3 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:19.683927 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:19.691276 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be62468-fc66-46fd-955e-1ef5577145e3 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:19.726034 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e39bcc-74e4-43eb-a5b7-c4010c14cb53 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:19.726845 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e39bcc-74e4-43eb-a5b7-c4010c14cb53 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:19.726845 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e39bcc-74e4-43eb-a5b7-c4010c14cb53 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:19.728306 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 370/730] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:06:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:06:19.728737 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2770ae7-92b1-478f-9ed0-10ad4b420060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:19.729578 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2770ae7-92b1-478f-9ed0-10ad4b420060 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:19.729864 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2770ae7-92b1-478f-9ed0-10ad4b420060 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:19.730144 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2770ae7-92b1-478f-9ed0-10ad4b420060 None admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:19.733232 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2770ae7-92b1-478f-9ed0-10ad4b420060 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:19.733232 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2770ae7-92b1-478f-9ed0-10ad4b420060 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:19.733232 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2770ae7-92b1-478f-9ed0-10ad4b420060 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:19.736978 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b71dd82-f3b4-4ebf-97fd-894a058e8600 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:19.737268 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b71dd82-f3b4-4ebf-97fd-894a058e8600 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:19.737548 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b71dd82-f3b4-4ebf-97fd-894a058e8600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:19.755996 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 361/731] 10.222.0.23 () {56 vars in 1135 bytes} [Tue Jun 21 18:06:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:19.758196 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2770ae7-92b1-478f-9ed0-10ad4b420060 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:19.762238 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-3b71dd82-f3b4-4ebf-97fd-894a058e8600 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:06:19.763724 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-3b71dd82-f3b4-4ebf-97fd-894a058e8600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:06:19.846290 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 371/732] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:06:19 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:06:19.857713 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b6f10c-690a-4ff2-8c4b-464204916133 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:19.858331 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b6f10c-690a-4ff2-8c4b-464204916133 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:19.858824 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b6f10c-690a-4ff2-8c4b-464204916133 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:19.860809 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 372/733] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:06:19 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:06:19.865797 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0d6cd6-f6cb-4ad5-af3a-3e9bd47824bc None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:19.866359 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0d6cd6-f6cb-4ad5-af3a-3e9bd47824bc None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:19.866864 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0d6cd6-f6cb-4ad5-af3a-3e9bd47824bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:19.866971 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2770ae7-92b1-478f-9ed0-10ad4b420060 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:19.905124 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf5cbeb-77ea-4478-a074-ce6723e44b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:19.905784 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf5cbeb-77ea-4478-a074-ce6723e44b76 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:19.906065 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf5cbeb-77ea-4478-a074-ce6723e44b76 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:19.906338 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf5cbeb-77ea-4478-a074-ce6723e44b76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:19.906627 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-ab0d6cd6-f6cb-4ad5-af3a-3e9bd47824bc None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:06:19.908087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-ab0d6cd6-f6cb-4ad5-af3a-3e9bd47824bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:06:19.940815 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:19.982825 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf5cbeb-77ea-4478-a074-ce6723e44b76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:19.984294 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf5cbeb-77ea-4478-a074-ce6723e44b76 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:19.985431 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 373/734] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:06:19 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:06:19.987699 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0d6cd6-f6cb-4ad5-af3a-3e9bd47824bc None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:20.004146 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:20.110350 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 362/735] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:06:19 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 21 18:06:20.115164 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf5cbeb-77ea-4478-a074-ce6723e44b76 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:20.123515 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0d6cd6-f6cb-4ad5-af3a-3e9bd47824bc None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:20.180420 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85939a5-2505-4f89-ad99-4e19f8e5a2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:20.180420 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85939a5-2505-4f89-ad99-4e19f8e5a2cb None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:20.180420 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85939a5-2505-4f89-ad99-4e19f8e5a2cb None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:20.180420 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85939a5-2505-4f89-ad99-4e19f8e5a2cb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:20.185449 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:20.195364 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85939a5-2505-4f89-ad99-4e19f8e5a2cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:06:20.204708 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-f85939a5-2505-4f89-ad99-4e19f8e5a2cb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:06:20.204708 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:20.204708 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:20.204708 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:20.204708 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:20.204708 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:20.204708 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:20.204708 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:20.204708 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:20.204708 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:20.204708 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:20.204708 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:20.204708 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:06:20.204708 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:06:20.204708 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:06:20.204708 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:06:20.205744 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:06:20.205744 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:06:20.205744 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:06:20.205744 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:06:20.205744 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:20.205744 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:20.205744 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:20.205744 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:20.205744 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:20.205744 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:20.205744 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:20.205744 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:20.205744 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:20.205744 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:20.205744 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:20.205744 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:20.206725 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:20.206725 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:20.206725 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:20.206725 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:20.206725 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:06:20.206725 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:06:20.206725 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:06:20.206725 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:06:20.206725 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:06:20.206725 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:06:20.206725 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:06:20.206725 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:20.206725 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 374/736] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jun 21 18:06:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:20.207433 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85939a5-2505-4f89-ad99-4e19f8e5a2cb None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:20.245964 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf5cbeb-77ea-4478-a074-ce6723e44b76 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:20.289386 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cea212-c241-4fb0-81b4-a6eb2006cfc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:20.291426 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cea212-c241-4fb0-81b4-a6eb2006cfc2 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:20.291934 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cea212-c241-4fb0-81b4-a6eb2006cfc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:20.292465 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cea212-c241-4fb0-81b4-a6eb2006cfc2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:20.296868 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:20.305119 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cea212-c241-4fb0-81b4-a6eb2006cfc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:06:20.313669 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-e8cea212-c241-4fb0-81b4-a6eb2006cfc2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:06:20.313669 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:20.313669 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:20.313669 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:20.313669 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:20.313669 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:20.313669 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:20.313669 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:20.313669 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:20.313669 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:20.313669 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:20.313669 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:20.313669 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:06:20.313669 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:06:20.313669 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:06:20.313669 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:06:20.314628 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:06:20.314628 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:06:20.314628 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:06:20.314628 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:06:20.314628 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:20.314628 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:20.314628 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:20.314628 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:20.314628 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:20.314628 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:20.314628 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:20.314628 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:20.314628 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:20.314628 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:20.314628 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:20.314628 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:20.315594 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:20.315594 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:20.315594 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:20.315594 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:20.315594 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:06:20.315594 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:06:20.315594 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:06:20.315594 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:06:20.315594 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:06:20.315594 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:06:20.315594 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:06:20.315594 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:20.324643 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 363/737] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jun 21 18:06:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:20.325839 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cea212-c241-4fb0-81b4-a6eb2006cfc2 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:20.331849 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85939a5-2505-4f89-ad99-4e19f8e5a2cb None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:20.378333 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b00fbc5-4ac9-4774-9169-1fbda14df4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:20.379053 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b00fbc5-4ac9-4774-9169-1fbda14df4d6 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:20.379336 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b00fbc5-4ac9-4774-9169-1fbda14df4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:20.379673 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b00fbc5-4ac9-4774-9169-1fbda14df4d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:20.395031 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b00fbc5-4ac9-4774-9169-1fbda14df4d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:20.397211 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b00fbc5-4ac9-4774-9169-1fbda14df4d6 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:20.416605 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-9b00fbc5-4ac9-4774-9169-1fbda14df4d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:20.416605 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:20.416605 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:20.416605 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:20.416605 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:20.416605 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:20.416605 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:20.416605 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:20.416605 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:20.416605 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:20.416605 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:20.416605 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:20.416605 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:06:20.416605 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:06:20.416605 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:06:20.417622 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:06:20.417622 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:20.417622 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:20.417622 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:20.417622 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:20.417622 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:20.417622 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:20.417622 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:20.417622 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:20.417622 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:20.417622 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:20.417622 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:20.417622 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:20.417622 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:20.417622 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:20.417622 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:20.417622 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:20.418587 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:06:20.418587 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:06:20.418587 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:20.418587 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:20.418587 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 375/738] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:06:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:20.420243 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b00fbc5-4ac9-4774-9169-1fbda14df4d6 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:20.443273 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cea212-c241-4fb0-81b4-a6eb2006cfc2 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:20.457305 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:20.480910 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e979836-6f69-462f-8d49-895da284929d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:20.481600 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e979836-6f69-462f-8d49-895da284929d None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:20.481886 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e979836-6f69-462f-8d49-895da284929d None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:20.482168 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e979836-6f69-462f-8d49-895da284929d None admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:20.483002 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e979836-6f69-462f-8d49-895da284929d None admin] RBAC: Adding query filter params (name=service) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:20.483346 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e979836-6f69-462f-8d49-895da284929d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:20.484754 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e979836-6f69-462f-8d49-895da284929d None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:20.496606 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 364/739] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jun 21 18:06:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:20.497408 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e979836-6f69-462f-8d49-895da284929d None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:20.530633 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b00fbc5-4ac9-4774-9169-1fbda14df4d6 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:20.557678 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:20.568355 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b768956-607d-4033-932d-d6941d38c229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:20.569079 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b768956-607d-4033-932d-d6941d38c229 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:20.569340 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b768956-607d-4033-932d-d6941d38c229 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:20.569622 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b768956-607d-4033-932d-d6941d38c229 None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:20.570372 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b768956-607d-4033-932d-d6941d38c229 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:20.570680 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b768956-607d-4033-932d-d6941d38c229 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:20.572196 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b768956-607d-4033-932d-d6941d38c229 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:20.581119 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 376/740] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:06:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:20.582397 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf398ab-3919-4295-a6b5-86b02d8b0c4b None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:20.582685 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf398ab-3919-4295-a6b5-86b02d8b0c4b None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:20.582963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf398ab-3919-4295-a6b5-86b02d8b0c4b None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:20.588816 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 377/741] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:06:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:06:20.590142 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf398ab-3919-4295-a6b5-86b02d8b0c4b None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:20.615699 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e979836-6f69-462f-8d49-895da284929d None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:20.654397 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbed425e-83e2-4184-93af-423b73f80f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:20.655156 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed425e-83e2-4184-93af-423b73f80f53 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:20.655369 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed425e-83e2-4184-93af-423b73f80f53 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:20.655644 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed425e-83e2-4184-93af-423b73f80f53 None admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:20.656366 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbed425e-83e2-4184-93af-423b73f80f53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:20.656729 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbed425e-83e2-4184-93af-423b73f80f53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:20.658198 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbed425e-83e2-4184-93af-423b73f80f53 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:20.675315 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 365/742] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:06:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:20.675315 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbed425e-83e2-4184-93af-423b73f80f53 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:20.709382 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:20.713760 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf398ab-3919-4295-a6b5-86b02d8b0c4b None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:20.754795 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033a8c35-a7c7-4f62-9b74-6f820f0593a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:20.755625 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a8c35-a7c7-4f62-9b74-6f820f0593a2 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:20.755913 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a8c35-a7c7-4f62-9b74-6f820f0593a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:20.756192 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a8c35-a7c7-4f62-9b74-6f820f0593a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:20.768591 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033a8c35-a7c7-4f62-9b74-6f820f0593a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:20.770104 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033a8c35-a7c7-4f62-9b74-6f820f0593a2 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:20.791027 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-033a8c35-a7c7-4f62-9b74-6f820f0593a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:20.791027 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:20.791027 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:20.791027 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:20.791027 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:20.791027 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:20.791027 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:20.791027 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:20.791027 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:20.791027 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:20.791027 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:20.791027 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:20.791027 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:06:20.791027 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:06:20.791027 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:06:20.792681 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:06:20.792681 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:20.792681 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:20.792681 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:20.792681 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:20.792681 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:20.792681 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:20.792681 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:20.792681 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:20.792681 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:20.792681 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:20.792681 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:20.792681 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:20.792681 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:20.792681 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:20.792681 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:20.792681 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:20.793580 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:06:20.793580 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:06:20.793580 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:20.793580 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:20.793580 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 378/743] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:06:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:20.793580 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3220c2d-5ce3-4996-bb9b-957d90b17030 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:20.793922 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3220c2d-5ce3-4996-bb9b-957d90b17030 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:20.794445 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3220c2d-5ce3-4996-bb9b-957d90b17030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:20.800016 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbed425e-83e2-4184-93af-423b73f80f53 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:20.812569 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-f3220c2d-5ce3-4996-bb9b-957d90b17030 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:06:20.813803 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-f3220c2d-5ce3-4996-bb9b-957d90b17030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:06:20.841970 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1c08f2-9e90-4218-bd4e-10802264d02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:20.842811 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c08f2-9e90-4218-bd4e-10802264d02b None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:20.843099 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c08f2-9e90-4218-bd4e-10802264d02b None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:20.843379 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c08f2-9e90-4218-bd4e-10802264d02b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:20.848033 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:20.860197 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1c08f2-9e90-4218-bd4e-10802264d02b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:20.862020 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1c08f2-9e90-4218-bd4e-10802264d02b None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:20.882663 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-8c1c08f2-9e90-4218-bd4e-10802264d02b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:20.882663 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:20.882663 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:20.882663 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:20.882663 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:20.882663 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:20.882663 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:20.882663 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:20.882663 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:20.882663 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:20.882663 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:20.882663 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:20.882663 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:06:20.882663 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:06:20.882663 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:06:20.883651 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:06:20.883651 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:20.883651 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:20.883651 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:20.883651 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:20.883651 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:20.883651 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:20.883651 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:20.883651 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:20.883651 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:20.883651 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:20.883651 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:20.883651 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:20.883651 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:20.883651 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:20.883651 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:20.883651 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:20.884646 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:06:20.884646 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:06:20.884646 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:20.884646 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:20.884646 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 366/744] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:06:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:20.885237 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1c08f2-9e90-4218-bd4e-10802264d02b None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:20.904112 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 379/745] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:06:20 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:06:20.905276 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3220c2d-5ce3-4996-bb9b-957d90b17030 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:20.987217 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:21.001596 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1c08f2-9e90-4218-bd4e-10802264d02b None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:21.013074 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3220c2d-5ce3-4996-bb9b-957d90b17030 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:21.037281 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22af03c-4ee7-4955-beaf-e207100ceb0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:21.038078 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22af03c-4ee7-4955-beaf-e207100ceb0b None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:21.038359 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22af03c-4ee7-4955-beaf-e207100ceb0b None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:21.038633 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22af03c-4ee7-4955-beaf-e207100ceb0b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:21.046013 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22af03c-4ee7-4955-beaf-e207100ceb0b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:21.047451 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22af03c-4ee7-4955-beaf-e207100ceb0b None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:21.052983 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724ddc28-6546-41dd-aa59-e6a433dfaada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:21.053871 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724ddc28-6546-41dd-aa59-e6a433dfaada None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:21.054337 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724ddc28-6546-41dd-aa59-e6a433dfaada None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:21.054793 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724ddc28-6546-41dd-aa59-e6a433dfaada None admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:21.055748 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724ddc28-6546-41dd-aa59-e6a433dfaada None admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:21.056241 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724ddc28-6546-41dd-aa59-e6a433dfaada None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:21.057814 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724ddc28-6546-41dd-aa59-e6a433dfaada None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:21.061003 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-f22af03c-4ee7-4955-beaf-e207100ceb0b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:06:21.061003 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:21.061003 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:21.061003 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:21.061003 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:21.061003 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:21.061003 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:21.061003 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:21.061003 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:21.061003 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:21.061003 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:21.061003 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:21.061003 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:06:21.061003 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:06:21.061003 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:06:21.061909 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:06:21.061909 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:21.061909 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:21.061909 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:21.061909 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:21.061909 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:21.061909 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:21.061909 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:21.061909 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:21.061909 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:21.061909 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:21.061909 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:21.061909 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:21.061909 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:21.061909 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:21.061909 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:21.061909 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:21.062914 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:06:21.062914 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:06:21.062914 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:06:21.062914 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:06:21.062914 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:06:21.062914 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:06:21.062914 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:06:21.062914 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:21.062914 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 367/746] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jun 21 18:06:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:21.063410 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22af03c-4ee7-4955-beaf-e207100ceb0b None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:21.073754 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 380/747] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:06:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:21.075004 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724ddc28-6546-41dd-aa59-e6a433dfaada None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:21.107786 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:21.181257 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22af03c-4ee7-4955-beaf-e207100ceb0b None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:21.184803 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724ddc28-6546-41dd-aa59-e6a433dfaada None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:21.216919 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d577a02-1fac-4adb-bada-00ab47782118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:21.217595 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d577a02-1fac-4adb-bada-00ab47782118 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:21.217980 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d577a02-1fac-4adb-bada-00ab47782118 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:21.218351 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d577a02-1fac-4adb-bada-00ab47782118 None admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:21.219169 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d577a02-1fac-4adb-bada-00ab47782118 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:21.219517 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d577a02-1fac-4adb-bada-00ab47782118 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:21.220725 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b58aea9-8f65-4d7a-9bee-ca074719accf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:21.221010 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d577a02-1fac-4adb-bada-00ab47782118 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:21.221389 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b58aea9-8f65-4d7a-9bee-ca074719accf None admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:21.221670 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b58aea9-8f65-4d7a-9bee-ca074719accf None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:21.221945 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b58aea9-8f65-4d7a-9bee-ca074719accf None admin] PATH_INFO: `/v3/services` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:21.222669 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b58aea9-8f65-4d7a-9bee-ca074719accf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:21.224288 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b58aea9-8f65-4d7a-9bee-ca074719accf None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:21.234494 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 368/748] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:06:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:21.235171 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d577a02-1fac-4adb-bada-00ab47782118 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:21.247788 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:21.312423 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 381/749] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Jun 21 18:06:21 2022] POST /identity/v3/services => generated 240 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:06:21.312423 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b58aea9-8f65-4d7a-9bee-ca074719accf None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:21.352897 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d577a02-1fac-4adb-bada-00ab47782118 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:21.401572 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf87dd9-8d82-449e-b2d4-5bb80199c97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:21.402379 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf87dd9-8d82-449e-b2d4-5bb80199c97b None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:21.402667 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf87dd9-8d82-449e-b2d4-5bb80199c97b None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:21.402950 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf87dd9-8d82-449e-b2d4-5bb80199c97b None admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:21.403751 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf87dd9-8d82-449e-b2d4-5bb80199c97b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:21.404064 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf87dd9-8d82-449e-b2d4-5bb80199c97b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:21.405810 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf87dd9-8d82-449e-b2d4-5bb80199c97b None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:21.420443 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 369/750] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jun 21 18:06:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:21.421330 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf87dd9-8d82-449e-b2d4-5bb80199c97b None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:21.441194 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:21.457740 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b58aea9-8f65-4d7a-9bee-ca074719accf None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:21.509820 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:21.520868 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04538dc3-538f-4326-86e6-22ad7de58c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:21.521703 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04538dc3-538f-4326-86e6-22ad7de58c83 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:21.521993 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04538dc3-538f-4326-86e6-22ad7de58c83 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:21.523371 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04538dc3-538f-4326-86e6-22ad7de58c83 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:21.543212 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04538dc3-538f-4326-86e6-22ad7de58c83 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:21.546080 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04538dc3-538f-4326-86e6-22ad7de58c83 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:21.563213 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf87dd9-8d82-449e-b2d4-5bb80199c97b None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:21.566134 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-04538dc3-538f-4326-86e6-22ad7de58c83 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 18:06:21.566134 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:21.566134 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:21.566134 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:21.566134 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:21.566134 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:21.566134 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:21.566134 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:21.566134 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:21.566134 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:21.566134 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:21.566134 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:21.566134 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:06:21.566134 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:06:21.566134 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:06:21.566134 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:06:21.567099 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:21.567099 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:21.567099 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:06:21.567099 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:06:21.567099 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:06:21.567099 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:06:21.567099 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:21.567099 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:21.567099 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:21.567099 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:21.567099 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:21.567099 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:21.567099 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:21.567099 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:21.567099 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:21.567099 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:21.567099 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:21.567099 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:21.568057 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:21.568057 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:21.568057 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:06:21.568057 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:06:21.568057 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:06:21.568057 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:06:21.568057 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:06:21.568057 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:06:21.568057 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 18:06:21.568057 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:21.568057 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 382/751] 10.222.0.23 () {56 vars in 1117 bytes} [Tue Jun 21 18:06:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:21.568692 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04538dc3-538f-4326-86e6-22ad7de58c83 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:21.615060 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c11245-7096-411e-a021-51a7c3a9cd90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:21.615874 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c11245-7096-411e-a021-51a7c3a9cd90 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:21.616168 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c11245-7096-411e-a021-51a7c3a9cd90 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:21.616480 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c11245-7096-411e-a021-51a7c3a9cd90 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:21.626607 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c11245-7096-411e-a021-51a7c3a9cd90 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:21.628365 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c11245-7096-411e-a021-51a7c3a9cd90 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:21.644261 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-12c11245-7096-411e-a021-51a7c3a9cd90 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 21 18:06:21.644261 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:21.644261 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:21.644261 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:21.644261 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:21.644261 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:21.644261 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:21.644261 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:21.644261 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:21.644261 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:21.644261 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:21.644261 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:21.644261 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:06:21.644261 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:06:21.644261 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:06:21.644261 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:06:21.645311 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:21.645311 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:21.645311 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:06:21.645311 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:06:21.645311 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:06:21.645311 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:06:21.645311 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:21.645311 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:21.645311 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:21.645311 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:21.645311 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:21.645311 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:21.645311 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:21.645311 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:21.645311 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:21.645311 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:21.645311 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:21.645311 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:21.646267 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:21.646267 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:21.646267 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:06:21.646267 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:06:21.646267 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:06:21.646267 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:06:21.646267 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:06:21.646267 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:06:21.646267 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 21 18:06:21.646267 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:21.646267 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 370/752] 10.222.0.23 () {56 vars in 1093 bytes} [Tue Jun 21 18:06:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:21.646865 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c11245-7096-411e-a021-51a7c3a9cd90 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:21.692857 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04538dc3-538f-4326-86e6-22ad7de58c83 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:21.733994 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcc3d95-2f94-4bbe-a337-d16fc2ff4572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:21.734868 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc3d95-2f94-4bbe-a337-d16fc2ff4572 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:21.735463 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc3d95-2f94-4bbe-a337-d16fc2ff4572 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:21.735805 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc3d95-2f94-4bbe-a337-d16fc2ff4572 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:21.737350 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcc3d95-2f94-4bbe-a337-d16fc2ff4572 None admin] RBAC: Adding query filter params (role.id=745ed9065a8143e384ebbd694b29635c, scope.project.id=2a4699d1ec70456da5f8e0aa4b1439b7, user.id=0e0d25565f1543d9a7577288fc09119f) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:21.737914 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcc3d95-2f94-4bbe-a337-d16fc2ff4572 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:21.739788 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcc3d95-2f94-4bbe-a337-d16fc2ff4572 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:21.749456 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 383/753] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jun 21 18:06:21 2022] GET /identity/v3/role_assignments?user.id=0e0d25565f1543d9a7577288fc09119f&scope.project.id=2a4699d1ec70456da5f8e0aa4b1439b7&role.id=745ed9065a8143e384ebbd694b29635c => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:21.750806 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcc3d95-2f94-4bbe-a337-d16fc2ff4572 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:21.787711 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:21.811460 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c11245-7096-411e-a021-51a7c3a9cd90 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:21.847593 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:21.872325 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b32c5d-e04f-4fd1-934d-774fbf922012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:21.873346 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b32c5d-e04f-4fd1-934d-774fbf922012 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:21.873634 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b32c5d-e04f-4fd1-934d-774fbf922012 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:21.873911 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b32c5d-e04f-4fd1-934d-774fbf922012 None admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:21.874774 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b32c5d-e04f-4fd1-934d-774fbf922012 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:21.875083 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b32c5d-e04f-4fd1-934d-774fbf922012 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:21.876949 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b32c5d-e04f-4fd1-934d-774fbf922012 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:21.912182 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 371/754] 10.222.0.23 () {56 vars in 1149 bytes} [Tue Jun 21 18:06:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:21.912182 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa249b-7559-49b4-a6c9-7772c27fa7ab None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:21.912182 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa249b-7559-49b4-a6c9-7772c27fa7ab None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:21.912182 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa249b-7559-49b4-a6c9-7772c27fa7ab None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:21.916985 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 372/755] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:06:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:06:21.917627 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaa249b-7559-49b4-a6c9-7772c27fa7ab None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:21.937620 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcc3d95-2f94-4bbe-a337-d16fc2ff4572 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:21.975228 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc439d02-b146-461e-a906-bee6eb4542f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:21.975946 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc439d02-b146-461e-a906-bee6eb4542f1 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:21.976235 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc439d02-b146-461e-a906-bee6eb4542f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:21.976632 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc439d02-b146-461e-a906-bee6eb4542f1 None admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:21.977489 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc439d02-b146-461e-a906-bee6eb4542f1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:21.978472 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc439d02-b146-461e-a906-bee6eb4542f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:21.979824 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc439d02-b146-461e-a906-bee6eb4542f1 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:21.999237 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 384/756] 10.222.0.23 () {56 vars in 1133 bytes} [Tue Jun 21 18:06:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:22.000577 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d98a4e-98c6-4c10-8e52-e6673f45400b None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:22.000925 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d98a4e-98c6-4c10-8e52-e6673f45400b None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:22.001149 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d98a4e-98c6-4c10-8e52-e6673f45400b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:22.018853 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-53d98a4e-98c6-4c10-8e52-e6673f45400b None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:06:22.020062 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-53d98a4e-98c6-4c10-8e52-e6673f45400b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:06:22.032997 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaa249b-7559-49b4-a6c9-7772c27fa7ab None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:22.072375 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc16f1a1-bcea-49d1-947f-219f82d8108c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:22.073128 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16f1a1-bcea-49d1-947f-219f82d8108c None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:22.073508 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16f1a1-bcea-49d1-947f-219f82d8108c None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:22.073879 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16f1a1-bcea-49d1-947f-219f82d8108c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:22.109563 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:22.124438 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 385/757] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:06:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:06:22.124438 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d98a4e-98c6-4c10-8e52-e6673f45400b None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:22.146754 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:22.173561 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc16f1a1-bcea-49d1-947f-219f82d8108c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:22.175393 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc16f1a1-bcea-49d1-947f-219f82d8108c None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:22.241332 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d98a4e-98c6-4c10-8e52-e6673f45400b None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:22.279419 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b0776b-f57c-4ce0-8ed0-843e61076b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:22.281219 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0776b-f57c-4ce0-8ed0-843e61076b08 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:22.281709 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0776b-f57c-4ce0-8ed0-843e61076b08 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:22.282286 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0776b-f57c-4ce0-8ed0-843e61076b08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:22.296513 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b0776b-f57c-4ce0-8ed0-843e61076b08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:22.299024 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b0776b-f57c-4ce0-8ed0-843e61076b08 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:22.306219 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 373/758] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:06:21 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 18:06:22.307100 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc16f1a1-bcea-49d1-947f-219f82d8108c None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:22.322565 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-79b0776b-f57c-4ce0-8ed0-843e61076b08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:22.322565 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:22.322565 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:22.322565 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:22.322565 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:22.322565 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:22.322565 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:22.322565 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:22.322565 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:22.322565 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:22.322565 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:22.322565 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:22.322565 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:06:22.322565 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:06:22.322565 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:06:22.323542 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:06:22.323542 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:22.323542 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:22.323542 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:22.323542 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:22.323542 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:22.323542 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:22.323542 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:22.323542 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:22.323542 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:22.323542 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:22.323542 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:22.323542 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:22.323542 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:22.323542 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:22.323542 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:22.323542 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:22.325147 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:06:22.325147 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:06:22.325147 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:22.325147 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:22.325147 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 386/759] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:06:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:22.325147 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b0776b-f57c-4ce0-8ed0-843e61076b08 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:22.398350 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:22.415850 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:22.444189 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc16f1a1-bcea-49d1-947f-219f82d8108c None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:22.459805 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b0776b-f57c-4ce0-8ed0-843e61076b08 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:22.492998 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d55ab2a-9b70-46ef-8b36-98011794a679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:22.495079 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55ab2a-9b70-46ef-8b36-98011794a679 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:22.495593 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55ab2a-9b70-46ef-8b36-98011794a679 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:22.496085 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55ab2a-9b70-46ef-8b36-98011794a679 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:22.496976 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d55ab2a-9b70-46ef-8b36-98011794a679 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:22.498619 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d55ab2a-9b70-46ef-8b36-98011794a679 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:22.509598 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12dfc76-7880-46c6-9f18-483ef126cd9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:22.510425 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12dfc76-7880-46c6-9f18-483ef126cd9e None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:22.510713 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12dfc76-7880-46c6-9f18-483ef126cd9e None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:22.510994 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12dfc76-7880-46c6-9f18-483ef126cd9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:22.513617 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-4d55ab2a-9b70-46ef-8b36-98011794a679 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 21 18:06:22.513617 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:22.513617 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:06:22.513617 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:06:22.513617 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:06:22.513617 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:06:22.513617 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:06:22.513617 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:06:22.513617 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 21 18:06:22.513617 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:22.513617 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:06:22.513617 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:22.513617 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:22.513617 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:22.513617 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:22.513617 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:22.514553 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:22.514553 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:22.514553 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:22.514553 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:22.514553 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:22.514553 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:22.514553 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:22.514553 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:06:22.514553 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:06:22.514553 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:06:22.514553 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:06:22.514553 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:22.514553 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:22.514553 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:22.514553 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:22.514553 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:22.515434 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:22.515434 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:22.515434 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:22.515434 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:22.515434 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:22.515434 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:22.515434 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:22.515434 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:22.515434 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:22.515434 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:22.515434 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:22.515434 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:06:22.515434 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:06:22.515434 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 21 18:06:22.515434 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:22.522789 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12dfc76-7880-46c6-9f18-483ef126cd9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:22.523105 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 374/760] 10.222.0.23 () {56 vars in 1129 bytes} [Tue Jun 21 18:06:22 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:22.523273 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12dfc76-7880-46c6-9f18-483ef126cd9e None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:22.524300 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d55ab2a-9b70-46ef-8b36-98011794a679 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:22.543341 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-f12dfc76-7880-46c6-9f18-483ef126cd9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:22.543341 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:22.543341 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:22.543341 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:22.543341 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:22.543341 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:22.543341 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:22.543341 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:22.543341 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:22.543341 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:22.543341 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:22.543341 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:22.543341 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:06:22.543341 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:06:22.543341 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:06:22.544321 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:06:22.544321 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:22.544321 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:22.544321 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:22.544321 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:22.544321 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:22.544321 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:22.544321 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:22.544321 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:22.544321 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:22.544321 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:22.544321 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:22.544321 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:22.544321 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:22.544321 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:22.544321 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:22.544321 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:22.545911 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:06:22.545911 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:06:22.545911 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:22.545911 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:22.545911 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 387/761] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:06:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:22.545911 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12dfc76-7880-46c6-9f18-483ef126cd9e None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:22.641840 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d55ab2a-9b70-46ef-8b36-98011794a679 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:22.658065 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12dfc76-7880-46c6-9f18-483ef126cd9e None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:22.660504 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:22.678709 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8729f205-f041-4099-936c-d147bce7f6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:22.679838 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8729f205-f041-4099-936c-d147bce7f6c2 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:22.680352 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8729f205-f041-4099-936c-d147bce7f6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:22.680887 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8729f205-f041-4099-936c-d147bce7f6c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:22.681810 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8729f205-f041-4099-936c-d147bce7f6c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:22.682326 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8729f205-f041-4099-936c-d147bce7f6c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:22.683941 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8729f205-f041-4099-936c-d147bce7f6c2 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:22.694222 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2d0dcd-f4b0-47d7-9616-b3a5cacb2291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:22.694954 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2d0dcd-f4b0-47d7-9616-b3a5cacb2291 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:22.695176 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2d0dcd-f4b0-47d7-9616-b3a5cacb2291 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:22.695460 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2d0dcd-f4b0-47d7-9616-b3a5cacb2291 None admin] PATH_INFO: `/v3/services` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:22.696175 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2d0dcd-f4b0-47d7-9616-b3a5cacb2291 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:22.697084 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2d0dcd-f4b0-47d7-9616-b3a5cacb2291 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:22.698655 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2d0dcd-f4b0-47d7-9616-b3a5cacb2291 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:22.699146 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 375/762] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:06:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:22.700293 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8729f205-f041-4099-936c-d147bce7f6c2 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:22.708067 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 388/763] 10.222.0.23 () {56 vars in 1124 bytes} [Tue Jun 21 18:06:22 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:22.708067 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2d0dcd-f4b0-47d7-9616-b3a5cacb2291 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:22.709146 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:22.813325 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8729f205-f041-4099-936c-d147bce7f6c2 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:22.816782 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2d0dcd-f4b0-47d7-9616-b3a5cacb2291 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:22.854268 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8471139-911b-4c60-ac10-e6df5ed98512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:22.854268 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8471139-911b-4c60-ac10-e6df5ed98512 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:22.854568 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8471139-911b-4c60-ac10-e6df5ed98512 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:22.854641 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8471139-911b-4c60-ac10-e6df5ed98512 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:22.857095 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815a4453-eb17-47bc-b5f7-f5d6da0411c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:22.858057 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a4453-eb17-47bc-b5f7-f5d6da0411c3 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:22.858571 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a4453-eb17-47bc-b5f7-f5d6da0411c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:22.859046 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a4453-eb17-47bc-b5f7-f5d6da0411c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:22.860015 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815a4453-eb17-47bc-b5f7-f5d6da0411c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:22.860545 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815a4453-eb17-47bc-b5f7-f5d6da0411c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:22.861960 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8471139-911b-4c60-ac10-e6df5ed98512 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:22.862365 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815a4453-eb17-47bc-b5f7-f5d6da0411c3 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:22.863393 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8471139-911b-4c60-ac10-e6df5ed98512 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:22.876945 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 376/764] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:06:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:22.878230 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815a4453-eb17-47bc-b5f7-f5d6da0411c3 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:22.878977 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-d8471139-911b-4c60-ac10-e6df5ed98512 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:06:22.878977 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:22.878977 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:22.878977 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:22.878977 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:22.878977 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:22.878977 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:22.878977 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:22.878977 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:22.878977 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:22.878977 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:22.878977 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:22.878977 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:06:22.878977 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:06:22.878977 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:06:22.879862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:06:22.879862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:22.879862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:22.879862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:22.879862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:22.879862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:22.879862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:22.879862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:22.879862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:22.879862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:22.879862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:22.879862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:22.879862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:22.879862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:22.879862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:22.879862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:22.879862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:22.881214 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:06:22.881214 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:06:22.881214 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:06:22.881214 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:06:22.881214 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:06:22.881214 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:06:22.881214 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:06:22.881214 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:22.881214 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 389/765] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jun 21 18:06:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:22.893232 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8471139-911b-4c60-ac10-e6df5ed98512 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:22.960807 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:22.974307 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:23.024330 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815a4453-eb17-47bc-b5f7-f5d6da0411c3 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:23.029538 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8471139-911b-4c60-ac10-e6df5ed98512 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:23.070893 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011fd80-4587-4d54-8c34-61c34352842e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:23.072098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011fd80-4587-4d54-8c34-61c34352842e None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:23.072840 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011fd80-4587-4d54-8c34-61c34352842e None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:23.072840 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011fd80-4587-4d54-8c34-61c34352842e None admin] PATH_INFO: `/v3/services` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:23.073635 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0011fd80-4587-4d54-8c34-61c34352842e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:23.073964 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0011fd80-4587-4d54-8c34-61c34352842e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:23.075338 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0011fd80-4587-4d54-8c34-61c34352842e None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:23.083083 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c420749-5102-4952-a5e9-a478554c0853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:23.084336 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c420749-5102-4952-a5e9-a478554c0853 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:23.084880 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c420749-5102-4952-a5e9-a478554c0853 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:23.085473 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c420749-5102-4952-a5e9-a478554c0853 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:23.088257 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 377/766] 10.222.0.23 () {56 vars in 1124 bytes} [Tue Jun 21 18:06:22 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:23.089457 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011fd80-4587-4d54-8c34-61c34352842e None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:23.096983 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c420749-5102-4952-a5e9-a478554c0853 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:23.099028 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c420749-5102-4952-a5e9-a478554c0853 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:23.122247 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-6c420749-5102-4952-a5e9-a478554c0853 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:06:23.122247 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:23.122247 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:23.122247 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:23.122247 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:23.122247 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:23.122247 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:23.122247 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:23.122247 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:23.122247 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:23.122247 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:23.122247 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:23.122247 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:06:23.122247 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:06:23.122247 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:06:23.123231 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:06:23.123231 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:23.123231 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:23.123231 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:23.123231 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:23.123231 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:23.123231 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:23.123231 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:23.123231 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:23.123231 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:23.123231 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:23.123231 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:23.123231 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:23.123231 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:23.123231 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:23.123231 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:23.123231 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:23.124834 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:06:23.124834 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:06:23.124834 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:06:23.124834 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:06:23.124834 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:06:23.124834 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:06:23.124834 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:06:23.124834 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:23.131575 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 390/767] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jun 21 18:06:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:23.131575 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c420749-5102-4952-a5e9-a478554c0853 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:23.233643 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011fd80-4587-4d54-8c34-61c34352842e None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:23.256821 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:23.269353 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:23.275911 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c420749-5102-4952-a5e9-a478554c0853 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:23.278519 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df1156f-ff2d-4285-bc59-49cc58e9a617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:23.280017 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df1156f-ff2d-4285-bc59-49cc58e9a617 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:23.280017 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df1156f-ff2d-4285-bc59-49cc58e9a617 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:23.280017 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df1156f-ff2d-4285-bc59-49cc58e9a617 None admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:23.280722 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df1156f-ff2d-4285-bc59-49cc58e9a617 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:23.281041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df1156f-ff2d-4285-bc59-49cc58e9a617 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:23.282907 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df1156f-ff2d-4285-bc59-49cc58e9a617 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:23.302358 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 378/768] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jun 21 18:06:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:23.302358 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df1156f-ff2d-4285-bc59-49cc58e9a617 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:23.327535 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa92809-24cf-4544-a551-f81429361ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:23.329230 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa92809-24cf-4544-a551-f81429361ad7 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:23.330026 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa92809-24cf-4544-a551-f81429361ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:23.330508 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa92809-24cf-4544-a551-f81429361ad7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:23.332211 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa92809-24cf-4544-a551-f81429361ad7 None admin] RBAC: Adding query filter params (interface=public, service_id=c1ebfc259f46454b8258e3fa556e63fa, region_id=RegionOne) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:23.333057 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa92809-24cf-4544-a551-f81429361ad7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:23.335336 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa92809-24cf-4544-a551-f81429361ad7 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:23.347803 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 391/769] 10.222.0.23 () {56 vars in 1251 bytes} [Tue Jun 21 18:06:23 2022] GET /identity/v3/endpoints?service_id=c1ebfc259f46454b8258e3fa556e63fa&interface=public®ion_id=RegionOne => generated 194 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:23.349332 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa92809-24cf-4544-a551-f81429361ad7 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:23.435356 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df1156f-ff2d-4285-bc59-49cc58e9a617 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:23.465433 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa92809-24cf-4544-a551-f81429361ad7 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:23.484050 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe234fec-2e65-4845-b7ab-6f26a528f914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:23.484763 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe234fec-2e65-4845-b7ab-6f26a528f914 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:23.485058 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe234fec-2e65-4845-b7ab-6f26a528f914 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:23.485341 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe234fec-2e65-4845-b7ab-6f26a528f914 None admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:23.486262 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe234fec-2e65-4845-b7ab-6f26a528f914 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:23.486584 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe234fec-2e65-4845-b7ab-6f26a528f914 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:23.488163 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe234fec-2e65-4845-b7ab-6f26a528f914 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:23.507506 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 379/770] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jun 21 18:06:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:23.507506 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe234fec-2e65-4845-b7ab-6f26a528f914 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:23.516457 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963bd148-0b15-4cf6-9bc6-59afc6cff60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:23.517791 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963bd148-0b15-4cf6-9bc6-59afc6cff60f None admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:23.518315 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963bd148-0b15-4cf6-9bc6-59afc6cff60f None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:23.518795 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963bd148-0b15-4cf6-9bc6-59afc6cff60f None admin] PATH_INFO: `/v3/projects/2a4699d1ec70456da5f8e0aa4b1439b7/users/904255dd3c1f406eb10f6fc65f9cd098/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:23.522288 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963bd148-0b15-4cf6-9bc6-59afc6cff60f None admin] RBAC: Authorizing `identity:create_grant(project_id=2a4699d1ec70456da5f8e0aa4b1439b7, user_id=904255dd3c1f406eb10f6fc65f9cd098, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:23.525458 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963bd148-0b15-4cf6-9bc6-59afc6cff60f None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:23.564971 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:23.603812 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 392/771] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jun 21 18:06:23 2022] PUT /identity/v3/projects/2a4699d1ec70456da5f8e0aa4b1439b7/users/904255dd3c1f406eb10f6fc65f9cd098/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:06:23.613415 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963bd148-0b15-4cf6-9bc6-59afc6cff60f None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:23.620427 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe234fec-2e65-4845-b7ab-6f26a528f914 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:23.637892 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:23.667645 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c731b330-728c-41a3-ae84-406936983d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:23.668837 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c731b330-728c-41a3-ae84-406936983d2c None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:23.669349 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c731b330-728c-41a3-ae84-406936983d2c None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:23.669856 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c731b330-728c-41a3-ae84-406936983d2c None admin] PATH_INFO: `/v3/services` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:23.670845 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c731b330-728c-41a3-ae84-406936983d2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:23.674138 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c731b330-728c-41a3-ae84-406936983d2c None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:23.682351 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 380/772] 10.222.0.23 () {56 vars in 1087 bytes} [Tue Jun 21 18:06:23 2022] GET /identity/v3/services => generated 772 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:23.684210 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4d96d-6059-42be-805b-bd4e57538068 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:23.684773 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4d96d-6059-42be-805b-bd4e57538068 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:23.685268 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4d96d-6059-42be-805b-bd4e57538068 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:23.691594 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 381/773] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:06:23 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:06:23.693503 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5af3ed1-6cc2-463d-a5e9-f0326434c058 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:23.694086 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5af3ed1-6cc2-463d-a5e9-f0326434c058 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:23.694589 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5af3ed1-6cc2-463d-a5e9-f0326434c058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:23.715690 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-d5af3ed1-6cc2-463d-a5e9-f0326434c058 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:06:23.717436 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-d5af3ed1-6cc2-463d-a5e9-f0326434c058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:06:23.801659 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963bd148-0b15-4cf6-9bc6-59afc6cff60f None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:23.825916 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 382/774] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:06:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:06:23.845736 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341c3ef8-6f1f-460e-842e-e276253e3629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:23.846406 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c3ef8-6f1f-460e-842e-e276253e3629 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:23.846787 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c3ef8-6f1f-460e-842e-e276253e3629 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:23.847067 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c3ef8-6f1f-460e-842e-e276253e3629 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:23.847970 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341c3ef8-6f1f-460e-842e-e276253e3629 None admin] RBAC: Adding query filter params (role.id=acf4e26e0ceb41ddad002dd4edf6373c, scope.project.id=2a4699d1ec70456da5f8e0aa4b1439b7, user.id=fcfcd9f134e74b09bfdac337c81c8a07) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:23.848281 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341c3ef8-6f1f-460e-842e-e276253e3629 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:23.850168 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341c3ef8-6f1f-460e-842e-e276253e3629 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:23.855290 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5af3ed1-6cc2-463d-a5e9-f0326434c058 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:23.859397 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 393/775] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jun 21 18:06:23 2022] GET /identity/v3/role_assignments?user.id=fcfcd9f134e74b09bfdac337c81c8a07&scope.project.id=2a4699d1ec70456da5f8e0aa4b1439b7&role.id=acf4e26e0ceb41ddad002dd4edf6373c => generated 259 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:23.914960 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:23.981642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5af3ed1-6cc2-463d-a5e9-f0326434c058 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:24.026138 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff63d27-d449-4d30-b8a3-edd570601e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:24.027396 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff63d27-d449-4d30-b8a3-edd570601e37 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:24.027891 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff63d27-d449-4d30-b8a3-edd570601e37 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:24.028380 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff63d27-d449-4d30-b8a3-edd570601e37 None admin] PATH_INFO: `/v3/services/network` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:24.029385 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff63d27-d449-4d30-b8a3-edd570601e37 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:24.031038 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff63d27-d449-4d30-b8a3-edd570601e37 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:24.043721 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3732c-4331-4a40-b6b0-e884459e9867 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:24.044065 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3732c-4331-4a40-b6b0-e884459e9867 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:24.044356 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3732c-4331-4a40-b6b0-e884459e9867 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:24.044876 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-0ff63d27-d449-4d30-b8a3-edd570601e37 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 21 18:06:24.044876 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:24.044876 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:06:24.044876 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:06:24.044876 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:06:24.044876 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:06:24.044876 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:06:24.044876 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:06:24.044876 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 21 18:06:24.044876 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:24.044876 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:06:24.044876 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:24.044876 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:24.044876 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:24.044876 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:24.044876 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:24.044876 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:24.045885 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:24.045885 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:24.045885 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:24.045885 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:24.045885 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:24.045885 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:24.045885 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:06:24.045885 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:06:24.045885 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:06:24.045885 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:06:24.045885 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:24.045885 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:24.045885 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:24.045885 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:24.045885 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:24.045885 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:24.046730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:24.046730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:24.046730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:24.046730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:24.046730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:24.046730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:24.046730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:24.046730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:24.046730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:24.046730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:24.046730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:06:24.046730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:06:24.046730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 21 18:06:24.046730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:24.047529 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 394/776] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:06:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:06:24.047603 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 383/777] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jun 21 18:06:23 2022] GET /identity/v3/services/network => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:06:24.052269 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f3732c-4331-4a40-b6b0-e884459e9867 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:24.053704 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2a603f-d5e5-4e9e-a1c6-21c28421c7a5 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:24.053995 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2a603f-d5e5-4e9e-a1c6-21c28421c7a5 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:24.054275 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2a603f-d5e5-4e9e-a1c6-21c28421c7a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:24.072325 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-bd2a603f-d5e5-4e9e-a1c6-21c28421c7a5 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:06:24.073588 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-bd2a603f-d5e5-4e9e-a1c6-21c28421c7a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:06:24.161874 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:24.163920 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 384/778] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:06:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:06:24.187201 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2a603f-d5e5-4e9e-a1c6-21c28421c7a5 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:24.202446 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f3732c-4331-4a40-b6b0-e884459e9867 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:24.230048 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:24.266315 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d27f091-bd2e-4c9e-99cd-05e31cb18e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:24.267010 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d27f091-bd2e-4c9e-99cd-05e31cb18e8c None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:24.267300 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d27f091-bd2e-4c9e-99cd-05e31cb18e8c None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:24.267585 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d27f091-bd2e-4c9e-99cd-05e31cb18e8c None admin] PATH_INFO: `/v3/services` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:24.268312 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d27f091-bd2e-4c9e-99cd-05e31cb18e8c None admin] RBAC: Adding query filter params (name=network) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:24.268707 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d27f091-bd2e-4c9e-99cd-05e31cb18e8c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:24.270184 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d27f091-bd2e-4c9e-99cd-05e31cb18e8c None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:24.281989 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 395/779] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Jun 21 18:06:24 2022] GET /identity/v3/services?name=network => generated 124 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:06:24.288739 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d27f091-bd2e-4c9e-99cd-05e31cb18e8c None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:24.321292 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2a603f-d5e5-4e9e-a1c6-21c28421c7a5 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:24.360273 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8022c9-4389-4846-a43e-eb32db4cd601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:24.361374 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8022c9-4389-4846-a43e-eb32db4cd601 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:24.361847 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8022c9-4389-4846-a43e-eb32db4cd601 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:24.362275 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8022c9-4389-4846-a43e-eb32db4cd601 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:24.378167 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8022c9-4389-4846-a43e-eb32db4cd601 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:06:24.390870 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-4a8022c9-4389-4846-a43e-eb32db4cd601 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:06:24.390870 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:24.390870 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:24.390870 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:24.390870 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:24.390870 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:24.390870 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:24.390870 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:24.390870 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:24.390870 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:24.390870 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:24.390870 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:24.390870 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:06:24.390870 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:06:24.390870 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:06:24.390870 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:06:24.392192 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:06:24.392192 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:06:24.392192 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:06:24.392192 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:06:24.392192 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:24.392192 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:24.392192 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:24.392192 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:24.392192 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:24.392192 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:24.392192 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:24.392192 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:24.392192 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:24.392192 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:24.392192 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:24.392192 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:24.393284 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:24.393284 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:24.393284 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:24.393284 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:24.393284 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:06:24.393284 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:06:24.393284 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:06:24.393284 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:06:24.393284 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:06:24.393284 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:06:24.393284 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:06:24.393284 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:24.398166 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 385/780] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jun 21 18:06:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:24.407252 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8022c9-4389-4846-a43e-eb32db4cd601 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:24.506574 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d27f091-bd2e-4c9e-99cd-05e31cb18e8c None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:24.550491 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:24.558455 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8022c9-4389-4846-a43e-eb32db4cd601 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:24.585788 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682440ef-b6e2-48e2-976c-b5ff5dbb414f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:24.586487 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682440ef-b6e2-48e2-976c-b5ff5dbb414f None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:24.586777 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682440ef-b6e2-48e2-976c-b5ff5dbb414f None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:24.587152 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682440ef-b6e2-48e2-976c-b5ff5dbb414f None admin] PATH_INFO: `/v3/services` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:24.587894 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682440ef-b6e2-48e2-976c-b5ff5dbb414f None admin] RBAC: Adding query filter params (type=network) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:24.590432 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682440ef-b6e2-48e2-976c-b5ff5dbb414f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:24.590432 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682440ef-b6e2-48e2-976c-b5ff5dbb414f None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:24.598179 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f46bb6-9463-4626-8f38-52e452061083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:24.599117 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f46bb6-9463-4626-8f38-52e452061083 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:24.599437 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f46bb6-9463-4626-8f38-52e452061083 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:24.599717 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f46bb6-9463-4626-8f38-52e452061083 None admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:24.600500 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f46bb6-9463-4626-8f38-52e452061083 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:24.600814 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f46bb6-9463-4626-8f38-52e452061083 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:24.602254 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f46bb6-9463-4626-8f38-52e452061083 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:24.607024 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 396/781] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Jun 21 18:06:24 2022] GET /identity/v3/services?type=network => generated 350 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:06:24.612983 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 386/782] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:06:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:06:24.613150 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682440ef-b6e2-48e2-976c-b5ff5dbb414f None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:24.619724 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f46bb6-9463-4626-8f38-52e452061083 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:24.630397 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:24.736097 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f46bb6-9463-4626-8f38-52e452061083 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:24.772551 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7287aa-28c0-4cda-b47c-d3563776da89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:24.773283 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7287aa-28c0-4cda-b47c-d3563776da89 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:24.773569 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7287aa-28c0-4cda-b47c-d3563776da89 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:24.773849 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7287aa-28c0-4cda-b47c-d3563776da89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:24.787144 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7287aa-28c0-4cda-b47c-d3563776da89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:24.788564 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7287aa-28c0-4cda-b47c-d3563776da89 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:24.797896 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682440ef-b6e2-48e2-976c-b5ff5dbb414f None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:24.807032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-2f7287aa-28c0-4cda-b47c-d3563776da89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:24.807032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:24.807032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:24.807032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:24.807032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:24.807032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:24.807032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:24.807032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:24.807032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:24.807032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:24.807032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:24.807032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:24.807032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:06:24.807032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:06:24.807032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:06:24.808004 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:06:24.808004 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:24.808004 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:24.808004 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:24.808004 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:24.808004 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:24.808004 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:24.808004 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:24.808004 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:24.808004 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:24.808004 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:24.808004 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:24.808004 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:24.808004 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:24.808004 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:24.808004 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:24.808004 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:24.808987 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:06:24.808987 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:06:24.808987 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:24.808987 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:24.812773 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 387/783] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:06:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:06:24.814276 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7287aa-28c0-4cda-b47c-d3563776da89 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:24.828783 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:24.850347 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cbfce5-e660-4497-9630-f9aaf2ae0e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:24.851740 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cbfce5-e660-4497-9630-f9aaf2ae0e73 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:24.852241 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cbfce5-e660-4497-9630-f9aaf2ae0e73 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:24.853311 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cbfce5-e660-4497-9630-f9aaf2ae0e73 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:24.859018 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cbfce5-e660-4497-9630-f9aaf2ae0e73 None admin] RBAC: Adding query filter params (interface=public, service_id=6aefeb67dd154841a272aacbabb3efc5, region_id=RegionOne) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:24.859742 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cbfce5-e660-4497-9630-f9aaf2ae0e73 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:24.861407 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cbfce5-e660-4497-9630-f9aaf2ae0e73 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:24.878479 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 397/784] 10.222.0.23 () {56 vars in 1251 bytes} [Tue Jun 21 18:06:24 2022] GET /identity/v3/endpoints?service_id=6aefeb67dd154841a272aacbabb3efc5&interface=public®ion_id=RegionOne => generated 194 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:06:24.884813 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cbfce5-e660-4497-9630-f9aaf2ae0e73 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:24.927310 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7287aa-28c0-4cda-b47c-d3563776da89 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:24.965241 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a67a6-24c6-4285-8d5f-08a18a0e715f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:24.966010 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a67a6-24c6-4285-8d5f-08a18a0e715f None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:24.966297 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a67a6-24c6-4285-8d5f-08a18a0e715f None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:24.966578 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a67a6-24c6-4285-8d5f-08a18a0e715f None admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:24.967337 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8a67a6-24c6-4285-8d5f-08a18a0e715f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:24.967650 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8a67a6-24c6-4285-8d5f-08a18a0e715f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:24.969055 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8a67a6-24c6-4285-8d5f-08a18a0e715f None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:24.982513 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 388/785] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:06:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:06:24.987994 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a67a6-24c6-4285-8d5f-08a18a0e715f None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:25.065900 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:25.066289 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:25.100528 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a67a6-24c6-4285-8d5f-08a18a0e715f None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:25.131631 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cbfce5-e660-4497-9630-f9aaf2ae0e73 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:25.143934 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aefba82-f963-4c7e-8137-1b4c01ebc5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:25.144780 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aefba82-f963-4c7e-8137-1b4c01ebc5aa None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:25.145100 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aefba82-f963-4c7e-8137-1b4c01ebc5aa None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:25.145341 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aefba82-f963-4c7e-8137-1b4c01ebc5aa None admin] PATH_INFO: `/v3/users/swift` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:25.154293 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aefba82-f963-4c7e-8137-1b4c01ebc5aa None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:25.155883 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aefba82-f963-4c7e-8137-1b4c01ebc5aa None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:25.176503 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-9aefba82-f963-4c7e-8137-1b4c01ebc5aa None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 21 18:06:25.176503 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:25.176503 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:25.176503 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:25.176503 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:25.176503 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:25.176503 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:25.176503 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:25.176503 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:25.176503 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:25.176503 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:25.176503 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:25.176503 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:06:25.176503 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:06:25.176503 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:06:25.176503 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:06:25.177536 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:25.177536 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:25.177536 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:06:25.177536 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:06:25.177536 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:06:25.177536 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:06:25.177536 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:25.177536 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:25.177536 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:25.177536 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:25.177536 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:25.177536 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:25.177536 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:25.177536 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:25.177536 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:25.177536 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:25.177536 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:25.177536 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:25.178523 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:25.178523 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:25.178523 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:06:25.178523 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:06:25.178523 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:06:25.178523 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:06:25.178523 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:06:25.178523 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:06:25.178523 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 21 18:06:25.178523 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:25.179563 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 389/786] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jun 21 18:06:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:06:25.188138 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ebe749-bf61-4ef9-abcb-e1da4b754ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:25.189237 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ebe749-bf61-4ef9-abcb-e1da4b754ba6 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:25.189732 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ebe749-bf61-4ef9-abcb-e1da4b754ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:25.190214 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ebe749-bf61-4ef9-abcb-e1da4b754ba6 None admin] PATH_INFO: `/v3/services` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:25.190521 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aefba82-f963-4c7e-8137-1b4c01ebc5aa None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:25.191381 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ebe749-bf61-4ef9-abcb-e1da4b754ba6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:25.193050 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ebe749-bf61-4ef9-abcb-e1da4b754ba6 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:25.207592 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 398/787] 10.222.0.23 () {56 vars in 1087 bytes} [Tue Jun 21 18:06:24 2022] GET /identity/v3/services => generated 772 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:25.390969 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aefba82-f963-4c7e-8137-1b4c01ebc5aa None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:25.434748 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:25.443546 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378c90df-1bd8-4a08-9300-f77c1b06dfd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:25.453513 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378c90df-1bd8-4a08-9300-f77c1b06dfd8 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:25.453513 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378c90df-1bd8-4a08-9300-f77c1b06dfd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:25.453513 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378c90df-1bd8-4a08-9300-f77c1b06dfd8 None admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:25.453513 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378c90df-1bd8-4a08-9300-f77c1b06dfd8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:25.453513 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378c90df-1bd8-4a08-9300-f77c1b06dfd8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:25.453513 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378c90df-1bd8-4a08-9300-f77c1b06dfd8 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:25.485521 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 390/788] 10.222.0.23 () {56 vars in 1135 bytes} [Tue Jun 21 18:06:25 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:25.495441 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ebe749-bf61-4ef9-abcb-e1da4b754ba6 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:25.624986 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ebe749-bf61-4ef9-abcb-e1da4b754ba6 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:25.684247 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:25.692667 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a68bf-6029-4b46-b14d-c2228cbea5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:25.694033 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a68bf-6029-4b46-b14d-c2228cbea5e8 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:25.694742 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a68bf-6029-4b46-b14d-c2228cbea5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:25.695444 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a68bf-6029-4b46-b14d-c2228cbea5e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:25.716564 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6a68bf-6029-4b46-b14d-c2228cbea5e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:25.720336 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6a68bf-6029-4b46-b14d-c2228cbea5e8 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:25.754808 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-1b6a68bf-6029-4b46-b14d-c2228cbea5e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:25.754808 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:25.754808 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:25.754808 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:25.754808 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:25.754808 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:25.754808 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:25.754808 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:25.754808 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:25.754808 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:25.754808 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:25.754808 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:25.754808 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:06:25.754808 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:06:25.754808 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:06:25.755994 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:06:25.755994 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:25.755994 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:25.755994 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:25.755994 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:25.755994 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:25.755994 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:25.755994 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:25.755994 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:25.755994 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:25.755994 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:25.755994 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:25.755994 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:25.755994 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:25.755994 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:25.755994 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:25.755994 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:25.757252 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:06:25.757252 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:06:25.757252 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:25.757252 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:25.759983 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 399/789] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:06:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:25.766073 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378c90df-1bd8-4a08-9300-f77c1b06dfd8 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:25.912224 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378c90df-1bd8-4a08-9300-f77c1b06dfd8 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:25.950794 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daead16d-574d-4f4b-a93e-325318e62e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:25.951424 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daead16d-574d-4f4b-a93e-325318e62e51 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:25.951713 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daead16d-574d-4f4b-a93e-325318e62e51 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:25.952029 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daead16d-574d-4f4b-a93e-325318e62e51 None admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:25.952872 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daead16d-574d-4f4b-a93e-325318e62e51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:25.953185 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daead16d-574d-4f4b-a93e-325318e62e51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:25.954738 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daead16d-574d-4f4b-a93e-325318e62e51 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:25.970472 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 391/790] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:06:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:25.977112 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a68bf-6029-4b46-b14d-c2228cbea5e8 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:26.159278 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a68bf-6029-4b46-b14d-c2228cbea5e8 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:26.215792 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494fcbfe-a86c-4814-a818-02c2433ab2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:26.216569 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494fcbfe-a86c-4814-a818-02c2433ab2e4 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:26.216861 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494fcbfe-a86c-4814-a818-02c2433ab2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:26.217144 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494fcbfe-a86c-4814-a818-02c2433ab2e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:26.231754 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494fcbfe-a86c-4814-a818-02c2433ab2e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:26.233293 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494fcbfe-a86c-4814-a818-02c2433ab2e4 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:26.257382 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-494fcbfe-a86c-4814-a818-02c2433ab2e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:06:26.257382 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:26.257382 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:26.257382 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:26.257382 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:26.257382 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:26.257382 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:26.257382 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:26.257382 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:26.257382 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:26.257382 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:26.257382 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:26.257382 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:06:26.257382 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:06:26.257382 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:06:26.258375 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:06:26.258375 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:26.258375 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:26.258375 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:26.258375 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:26.258375 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:26.258375 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:26.258375 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:26.258375 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:26.258375 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:26.258375 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:26.258375 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:26.258375 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:26.258375 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:26.258375 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:26.258375 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:26.258375 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:26.260188 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:06:26.260188 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:06:26.260188 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:06:26.260188 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:06:26.260188 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:06:26.260188 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:06:26.260188 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:06:26.260188 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:26.260188 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 400/791] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jun 21 18:06:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:26.265024 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daead16d-574d-4f4b-a93e-325318e62e51 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:26.300080 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc97ed-7fd7-4d70-a36d-c802aa0abf15 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:26.300382 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc97ed-7fd7-4d70-a36d-c802aa0abf15 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:26.300685 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc97ed-7fd7-4d70-a36d-c802aa0abf15 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:26.302315 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 401/792] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:06:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:06:26.309975 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1bbd0b-c7e6-47af-be07-6ebb03d62047 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:26.310335 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1bbd0b-c7e6-47af-be07-6ebb03d62047 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:26.310548 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1bbd0b-c7e6-47af-be07-6ebb03d62047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:26.341627 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-7e1bbd0b-c7e6-47af-be07-6ebb03d62047 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:06:26.342875 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-7e1bbd0b-c7e6-47af-be07-6ebb03d62047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:06:26.348993 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:26.404810 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:26.421676 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daead16d-574d-4f4b-a93e-325318e62e51 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:26.456412 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 402/793] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:06:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:06:26.456412 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a2c78a-1b40-4983-bc7a-40470d7bf1dc None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:26.456412 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a2c78a-1b40-4983-bc7a-40470d7bf1dc None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:26.456412 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a2c78a-1b40-4983-bc7a-40470d7bf1dc None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:26.456412 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 403/794] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:06:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:06:26.477327 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c4839b-37b1-4966-b60b-7b204e00b9bc None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:26.477925 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c4839b-37b1-4966-b60b-7b204e00b9bc None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:26.478073 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c4839b-37b1-4966-b60b-7b204e00b9bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:26.480026 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f3de1f-4688-4fa8-a3ff-990752eaf7d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:26.480990 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f3de1f-4688-4fa8-a3ff-990752eaf7d3 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:26.481425 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f3de1f-4688-4fa8-a3ff-990752eaf7d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:26.481847 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f3de1f-4688-4fa8-a3ff-990752eaf7d3 None admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:26.482808 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f3de1f-4688-4fa8-a3ff-990752eaf7d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:26.483288 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f3de1f-4688-4fa8-a3ff-990752eaf7d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:26.484970 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f3de1f-4688-4fa8-a3ff-990752eaf7d3 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:26.500511 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 392/795] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jun 21 18:06:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:26.501637 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f3de1f-4688-4fa8-a3ff-990752eaf7d3 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:26.515434 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-38c4839b-37b1-4966-b60b-7b204e00b9bc None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:06:26.517122 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-38c4839b-37b1-4966-b60b-7b204e00b9bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:06:26.643683 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 404/796] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:06:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:06:26.644829 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c4839b-37b1-4966-b60b-7b204e00b9bc None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:26.662860 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f3de1f-4688-4fa8-a3ff-990752eaf7d3 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:26.683176 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:26.728350 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402665ed-3ebf-44e3-ad64-3fea05d594b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:26.729123 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402665ed-3ebf-44e3-ad64-3fea05d594b2 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:26.729406 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402665ed-3ebf-44e3-ad64-3fea05d594b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:26.729819 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402665ed-3ebf-44e3-ad64-3fea05d594b2 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:26.730392 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402665ed-3ebf-44e3-ad64-3fea05d594b2 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:26.731853 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402665ed-3ebf-44e3-ad64-3fea05d594b2 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:26.746976 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-402665ed-3ebf-44e3-ad64-3fea05d594b2 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 21 18:06:26.746976 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:26.746976 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:06:26.746976 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:06:26.746976 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:06:26.746976 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:06:26.746976 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:06:26.746976 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:06:26.746976 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 21 18:06:26.746976 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:26.746976 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:06:26.746976 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:26.746976 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:26.746976 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:26.746976 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:26.746976 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:26.749130 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:26.749130 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:26.749130 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:26.749130 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:26.749130 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:26.749130 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:26.749130 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:26.749130 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:06:26.749130 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:06:26.749130 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:06:26.749130 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:06:26.749130 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:26.749130 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:26.749130 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:26.749130 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:26.749130 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:26.750752 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:26.750752 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:26.750752 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:26.750752 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:26.750752 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:26.750752 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:26.750752 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:26.750752 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:26.750752 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:26.750752 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:26.750752 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:26.750752 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:06:26.750752 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:06:26.750752 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 21 18:06:26.750752 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:26.750752 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 393/797] 10.222.0.23 () {56 vars in 1129 bytes} [Tue Jun 21 18:06:26 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:26.754134 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cff015-3ce2-448a-907b-076c72044744 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:26.754134 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cff015-3ce2-448a-907b-076c72044744 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:26.754134 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cff015-3ce2-448a-907b-076c72044744 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:26.754134 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 394/798] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:06:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:06:26.754134 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cff015-3ce2-448a-907b-076c72044744 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:26.820306 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c4839b-37b1-4966-b60b-7b204e00b9bc None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:26.828370 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:26.874022 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabf77e1-0a02-4175-82fd-7526aa1b48d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:26.875577 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf77e1-0a02-4175-82fd-7526aa1b48d4 None admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:26.876061 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf77e1-0a02-4175-82fd-7526aa1b48d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:26.876635 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf77e1-0a02-4175-82fd-7526aa1b48d4 None admin] PATH_INFO: `/v3/projects/2a4699d1ec70456da5f8e0aa4b1439b7/users/0e0d25565f1543d9a7577288fc09119f/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:26.880379 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabf77e1-0a02-4175-82fd-7526aa1b48d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a4699d1ec70456da5f8e0aa4b1439b7, user_id=0e0d25565f1543d9a7577288fc09119f, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:26.885891 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabf77e1-0a02-4175-82fd-7526aa1b48d4 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:26.929324 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cff015-3ce2-448a-907b-076c72044744 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:26.980530 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 405/799] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jun 21 18:06:26 2022] PUT /identity/v3/projects/2a4699d1ec70456da5f8e0aa4b1439b7/users/0e0d25565f1543d9a7577288fc09119f/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:06:26.980530 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a5a43-047a-433d-a4d7-08e5aa7a2ee7 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:26.980530 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a5a43-047a-433d-a4d7-08e5aa7a2ee7 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:26.980530 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a5a43-047a-433d-a4d7-08e5aa7a2ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:26.985354 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f550498-0b76-4718-b1f4-18dbdecc51f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:26.986825 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f550498-0b76-4718-b1f4-18dbdecc51f4 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:26.987436 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f550498-0b76-4718-b1f4-18dbdecc51f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:26.988026 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f550498-0b76-4718-b1f4-18dbdecc51f4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:27.010348 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f550498-0b76-4718-b1f4-18dbdecc51f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:06:27.012858 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-5f1a5a43-047a-433d-a4d7-08e5aa7a2ee7 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:06:27.015372 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-5f1a5a43-047a-433d-a4d7-08e5aa7a2ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:06:27.026384 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-3f550498-0b76-4718-b1f4-18dbdecc51f4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:06:27.026384 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:27.026384 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:27.026384 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:27.026384 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:27.026384 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:27.026384 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:27.026384 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:27.026384 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:27.026384 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:27.026384 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:27.026384 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:27.026384 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:06:27.026384 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:06:27.026384 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:06:27.026384 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:06:27.028266 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:06:27.028266 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:06:27.028266 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:06:27.028266 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:06:27.028266 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:27.028266 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:27.028266 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:27.028266 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:27.028266 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:27.028266 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:27.028266 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:27.028266 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:27.028266 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:27.028266 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:27.028266 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:27.028266 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:27.030026 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:27.030026 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:27.030026 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:27.030026 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:27.030026 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:06:27.030026 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:06:27.030026 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:06:27.030026 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:06:27.030026 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:06:27.030026 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:06:27.030026 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:06:27.030026 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:27.030026 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 395/800] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jun 21 18:06:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:27.031494 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f550498-0b76-4718-b1f4-18dbdecc51f4 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:27.093366 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:27.168259 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 406/801] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:06:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:06:27.169878 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1a5a43-047a-433d-a4d7-08e5aa7a2ee7 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:27.248952 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f550498-0b76-4718-b1f4-18dbdecc51f4 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:27.326163 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642dbb5f-9201-4498-8d34-a6b6fafe10ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:27.328011 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642dbb5f-9201-4498-8d34-a6b6fafe10ae None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:27.328738 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642dbb5f-9201-4498-8d34-a6b6fafe10ae None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:27.329406 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642dbb5f-9201-4498-8d34-a6b6fafe10ae None admin] PATH_INFO: `/v3/services` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:27.330831 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642dbb5f-9201-4498-8d34-a6b6fafe10ae None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:27.331539 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642dbb5f-9201-4498-8d34-a6b6fafe10ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:27.334904 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642dbb5f-9201-4498-8d34-a6b6fafe10ae None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:27.343707 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:27.346546 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 396/802] 10.222.0.23 () {56 vars in 1124 bytes} [Tue Jun 21 18:06:27 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:27.352630 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642dbb5f-9201-4498-8d34-a6b6fafe10ae None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:27.360994 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1a5a43-047a-433d-a4d7-08e5aa7a2ee7 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:27.423058 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce131714-02a9-40cd-8d29-19801d5e947c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:27.424536 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce131714-02a9-40cd-8d29-19801d5e947c None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:27.425203 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce131714-02a9-40cd-8d29-19801d5e947c None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:27.425850 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce131714-02a9-40cd-8d29-19801d5e947c None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:27.427364 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce131714-02a9-40cd-8d29-19801d5e947c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:27.428137 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce131714-02a9-40cd-8d29-19801d5e947c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:27.431629 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce131714-02a9-40cd-8d29-19801d5e947c None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:27.449660 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 407/803] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:06:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:27.451585 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce131714-02a9-40cd-8d29-19801d5e947c None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:27.566346 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:27.577800 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642dbb5f-9201-4498-8d34-a6b6fafe10ae None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:27.647789 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce131714-02a9-40cd-8d29-19801d5e947c None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:27.650434 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3494677-d287-4060-be28-dc2aa617c451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:27.652063 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3494677-d287-4060-be28-dc2aa617c451 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:27.652581 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3494677-d287-4060-be28-dc2aa617c451 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:27.653057 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3494677-d287-4060-be28-dc2aa617c451 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:27.676515 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3494677-d287-4060-be28-dc2aa617c451 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:06:27.691425 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-d3494677-d287-4060-be28-dc2aa617c451 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:06:27.691425 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:27.691425 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:27.691425 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:27.691425 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:27.691425 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:27.691425 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:27.691425 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:27.691425 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:27.691425 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:27.691425 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:27.691425 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:27.691425 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:06:27.691425 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:06:27.691425 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:06:27.691425 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:06:27.693317 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:06:27.693317 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:06:27.693317 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:06:27.693317 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:06:27.693317 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:27.693317 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:27.693317 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:27.693317 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:27.693317 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:27.693317 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:27.693317 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:27.693317 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:27.693317 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:27.693317 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:27.693317 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:27.693317 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:27.695021 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:27.695021 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:27.695021 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:27.695021 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:27.695021 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:06:27.695021 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:06:27.695021 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:06:27.695021 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:06:27.695021 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:06:27.695021 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:06:27.695021 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:06:27.695021 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:27.696330 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 397/804] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jun 21 18:06:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:27.697640 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3494677-d287-4060-be28-dc2aa617c451 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:27.719064 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bfe32b-9566-4bc3-80e4-c13f1416dbb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:27.719745 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bfe32b-9566-4bc3-80e4-c13f1416dbb6 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:27.720225 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bfe32b-9566-4bc3-80e4-c13f1416dbb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:27.720744 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bfe32b-9566-4bc3-80e4-c13f1416dbb6 None admin] PATH_INFO: `/v3/services` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:27.721992 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bfe32b-9566-4bc3-80e4-c13f1416dbb6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:27.722521 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bfe32b-9566-4bc3-80e4-c13f1416dbb6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:27.725294 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bfe32b-9566-4bc3-80e4-c13f1416dbb6 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:27.740433 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 408/805] 10.222.0.23 () {56 vars in 1124 bytes} [Tue Jun 21 18:06:27 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:27.740433 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bfe32b-9566-4bc3-80e4-c13f1416dbb6 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:27.779068 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:27.927748 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bfe32b-9566-4bc3-80e4-c13f1416dbb6 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:27.935958 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3494677-d287-4060-be28-dc2aa617c451 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:27.999236 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e4d8ca-988d-422f-9527-2a4a125e04aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:28.000908 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e4d8ca-988d-422f-9527-2a4a125e04aa None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:28.001591 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e4d8ca-988d-422f-9527-2a4a125e04aa None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:28.002076 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e4d8ca-988d-422f-9527-2a4a125e04aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:28.003987 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc65ceb0-1206-4d1d-9c16-a593f846f956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:28.006239 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc65ceb0-1206-4d1d-9c16-a593f846f956 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:28.007750 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc65ceb0-1206-4d1d-9c16-a593f846f956 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:28.008594 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc65ceb0-1206-4d1d-9c16-a593f846f956 None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:28.010785 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc65ceb0-1206-4d1d-9c16-a593f846f956 None admin] RBAC: Adding query filter params (name=service) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:28.011604 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc65ceb0-1206-4d1d-9c16-a593f846f956 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:28.014458 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc65ceb0-1206-4d1d-9c16-a593f846f956 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:28.022364 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e4d8ca-988d-422f-9527-2a4a125e04aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:28.025457 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e4d8ca-988d-422f-9527-2a4a125e04aa None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:28.032934 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 409/806] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jun 21 18:06:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:28.034832 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc65ceb0-1206-4d1d-9c16-a593f846f956 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:28.056920 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-58e4d8ca-988d-422f-9527-2a4a125e04aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:28.056920 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:28.056920 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:28.056920 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:28.056920 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:28.056920 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:28.056920 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:28.056920 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:28.056920 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:28.056920 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:28.056920 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:28.056920 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:28.056920 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:06:28.056920 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:06:28.056920 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:06:28.058687 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:06:28.058687 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:28.058687 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:28.058687 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:28.058687 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:28.058687 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:28.058687 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:28.058687 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:28.058687 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:28.058687 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:28.058687 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:28.058687 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:28.058687 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:28.058687 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:28.058687 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:28.058687 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:28.058687 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:28.061166 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:06:28.061166 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:06:28.061166 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:28.061166 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:28.064709 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 398/807] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:06:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:28.066549 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e4d8ca-988d-422f-9527-2a4a125e04aa None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:28.088149 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:28.189621 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:28.226073 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc65ceb0-1206-4d1d-9c16-a593f846f956 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:28.270643 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e4d8ca-988d-422f-9527-2a4a125e04aa None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:28.308459 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d431f719-838d-4040-80f9-416639efe32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:28.311139 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d431f719-838d-4040-80f9-416639efe32e None admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:28.312247 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d431f719-838d-4040-80f9-416639efe32e None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:28.313832 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d431f719-838d-4040-80f9-416639efe32e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:28.315954 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d431f719-838d-4040-80f9-416639efe32e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:28.320056 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d431f719-838d-4040-80f9-416639efe32e None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:28.322152 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e493ec-010d-492e-9744-19482229a263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:28.323081 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e493ec-010d-492e-9744-19482229a263 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:28.323454 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e493ec-010d-492e-9744-19482229a263 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:28.323852 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e493ec-010d-492e-9744-19482229a263 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:28.337154 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e493ec-010d-492e-9744-19482229a263 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:28.339167 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e493ec-010d-492e-9744-19482229a263 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:28.361449 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-f5e493ec-010d-492e-9744-19482229a263 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:28.361449 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:28.361449 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:28.361449 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:28.361449 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:28.361449 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:28.361449 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:28.361449 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:28.361449 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:28.361449 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:28.361449 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:28.361449 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:28.361449 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:06:28.361449 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:06:28.361449 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:06:28.363126 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:06:28.363126 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:28.363126 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:28.363126 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:28.363126 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:28.363126 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:28.363126 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:28.363126 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:28.363126 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:28.363126 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:28.363126 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:28.363126 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:28.363126 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:28.363126 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:28.363126 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:28.363126 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:28.363126 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:28.365035 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:06:28.365035 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:06:28.365035 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:28.365035 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:28.367471 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 399/808] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:06:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:28.369150 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e493ec-010d-492e-9744-19482229a263 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:28.441325 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:28.451231 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 410/809] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Jun 21 18:06:28 2022] POST /identity/v3/endpoints => generated 346 bytes in 417 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:06:28.462155 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d431f719-838d-4040-80f9-416639efe32e None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:28.496659 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e493ec-010d-492e-9744-19482229a263 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:28.550855 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32b2ad7-69d7-49e0-9132-b08e44051a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:28.551539 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b2ad7-69d7-49e0-9132-b08e44051a62 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:28.551850 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b2ad7-69d7-49e0-9132-b08e44051a62 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:28.552098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b2ad7-69d7-49e0-9132-b08e44051a62 None admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:28.552849 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32b2ad7-69d7-49e0-9132-b08e44051a62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:28.553146 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32b2ad7-69d7-49e0-9132-b08e44051a62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:28.554563 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32b2ad7-69d7-49e0-9132-b08e44051a62 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:28.575526 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 400/810] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:06:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:28.578748 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb1efc-1671-4eca-9079-eaf923e06def None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:28.579697 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb1efc-1671-4eca-9079-eaf923e06def None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:28.580974 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb1efc-1671-4eca-9079-eaf923e06def None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:28.591503 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 401/811] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:06:28 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:06:28.593734 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddb1efc-1671-4eca-9079-eaf923e06def None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:28.645040 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d431f719-838d-4040-80f9-416639efe32e None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:28.698065 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6bbfe8-b306-411d-b64a-9d9165f13f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:28.699720 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6bbfe8-b306-411d-b64a-9d9165f13f87 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:28.700508 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6bbfe8-b306-411d-b64a-9d9165f13f87 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:28.701276 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6bbfe8-b306-411d-b64a-9d9165f13f87 None admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:28.702913 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6bbfe8-b306-411d-b64a-9d9165f13f87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:28.704898 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6bbfe8-b306-411d-b64a-9d9165f13f87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:28.707862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6bbfe8-b306-411d-b64a-9d9165f13f87 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:28.720545 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:28.732238 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 411/812] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:06:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:28.734701 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a50153-4964-47a9-bc7e-b9c3fea32b72 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:28.735329 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a50153-4964-47a9-bc7e-b9c3fea32b72 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:28.735867 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a50153-4964-47a9-bc7e-b9c3fea32b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:28.759637 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddb1efc-1671-4eca-9079-eaf923e06def None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:28.770830 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-48a50153-4964-47a9-bc7e-b9c3fea32b72 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:06:28.773344 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-48a50153-4964-47a9-bc7e-b9c3fea32b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:06:28.797386 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8234217c-7c9e-4928-9ae2-d07413a6d87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:28.798353 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234217c-7c9e-4928-9ae2-d07413a6d87c None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:28.798735 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234217c-7c9e-4928-9ae2-d07413a6d87c None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:28.799111 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234217c-7c9e-4928-9ae2-d07413a6d87c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:28.807667 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8234217c-7c9e-4928-9ae2-d07413a6d87c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:28.809503 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8234217c-7c9e-4928-9ae2-d07413a6d87c None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:28.816761 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:28.826990 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-8234217c-7c9e-4928-9ae2-d07413a6d87c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 21 18:06:28.826990 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:28.826990 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:28.826990 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:28.826990 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:28.826990 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:28.826990 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:28.826990 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:28.826990 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:28.826990 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:28.826990 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:28.826990 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:28.826990 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:06:28.826990 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:06:28.826990 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:06:28.826990 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:06:28.833946 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:28.833946 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:28.833946 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:06:28.833946 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:06:28.833946 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:06:28.833946 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:06:28.833946 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:28.833946 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:28.833946 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:28.833946 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:28.833946 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:28.833946 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:28.833946 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:28.833946 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:28.833946 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:28.833946 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:28.833946 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:28.833946 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:28.835820 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:28.835820 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:28.835820 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:06:28.835820 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:06:28.835820 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:06:28.835820 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:06:28.835820 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:06:28.835820 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:06:28.835820 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 21 18:06:28.835820 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:28.835820 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 402/813] 10.222.0.23 () {56 vars in 1093 bytes} [Tue Jun 21 18:06:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:28.835820 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8234217c-7c9e-4928-9ae2-d07413a6d87c None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:28.940568 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 412/814] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:06:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:06:28.942536 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a50153-4964-47a9-bc7e-b9c3fea32b72 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:28.953705 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8234217c-7c9e-4928-9ae2-d07413a6d87c None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:28.991078 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2480d0-3e91-439f-a474-f1511f447f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:28.991937 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2480d0-3e91-439f-a474-f1511f447f4d None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:28.992242 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2480d0-3e91-439f-a474-f1511f447f4d None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:28.992530 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2480d0-3e91-439f-a474-f1511f447f4d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:29.009971 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2480d0-3e91-439f-a474-f1511f447f4d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:29.016256 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2480d0-3e91-439f-a474-f1511f447f4d None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:29.032978 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-fc2480d0-3e91-439f-a474-f1511f447f4d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 18:06:29.032978 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:29.032978 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:29.032978 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:29.032978 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:29.032978 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:29.032978 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:29.032978 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:29.032978 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:29.032978 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:29.032978 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:29.032978 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:29.032978 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:06:29.032978 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:06:29.032978 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:06:29.032978 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:06:29.034041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:29.034041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:29.034041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:06:29.034041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:06:29.034041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:06:29.034041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:06:29.034041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:29.034041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:29.034041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:29.034041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:29.034041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:29.034041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:29.034041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:29.034041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:29.034041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:29.034041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:29.034041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:29.034041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:29.035459 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:29.035459 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:29.035459 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:06:29.035459 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:06:29.035459 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:06:29.035459 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:06:29.035459 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:06:29.035459 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:06:29.035459 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 18:06:29.035459 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:29.035459 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 403/815] 10.222.0.23 () {56 vars in 1117 bytes} [Tue Jun 21 18:06:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:29.035459 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2480d0-3e91-439f-a474-f1511f447f4d None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:29.081513 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:29.132825 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a50153-4964-47a9-bc7e-b9c3fea32b72 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:29.167472 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2480d0-3e91-439f-a474-f1511f447f4d None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:29.171091 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:29.183492 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c462742-b341-46b4-b967-75b7afadcffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:29.184711 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c462742-b341-46b4-b967-75b7afadcffb None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:29.185204 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c462742-b341-46b4-b967-75b7afadcffb None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:29.185679 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c462742-b341-46b4-b967-75b7afadcffb None admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:29.186917 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c462742-b341-46b4-b967-75b7afadcffb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:29.187488 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c462742-b341-46b4-b967-75b7afadcffb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:29.189277 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c462742-b341-46b4-b967-75b7afadcffb None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:29.212563 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedd2111-c208-4b15-a571-dffadcd05367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:29.212943 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd2111-c208-4b15-a571-dffadcd05367 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:29.216774 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd2111-c208-4b15-a571-dffadcd05367 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:29.216774 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd2111-c208-4b15-a571-dffadcd05367 None admin] PATH_INFO: `/v3/services/network` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:29.216774 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedd2111-c208-4b15-a571-dffadcd05367 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:29.216774 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedd2111-c208-4b15-a571-dffadcd05367 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:29.217279 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 413/816] 10.222.0.23 () {56 vars in 1133 bytes} [Tue Jun 21 18:06:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:29.218496 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c462742-b341-46b4-b967-75b7afadcffb None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:29.230864 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-fedd2111-c208-4b15-a571-dffadcd05367 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 21 18:06:29.230864 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:29.230864 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:06:29.230864 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:06:29.230864 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:06:29.230864 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:06:29.230864 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:06:29.230864 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:06:29.230864 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 21 18:06:29.230864 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:29.230864 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:06:29.230864 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:29.230864 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:29.230864 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:29.230864 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:29.230864 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:29.230864 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:29.231951 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:29.231951 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:29.231951 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:29.231951 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:29.231951 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:29.231951 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:29.231951 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:06:29.231951 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:06:29.231951 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:06:29.231951 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:06:29.231951 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:29.231951 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:29.231951 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:29.231951 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:29.231951 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:29.231951 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:29.237747 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:29.237747 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:29.237747 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:29.237747 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:29.237747 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:29.237747 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:29.237747 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:29.237747 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:29.237747 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:29.237747 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:29.237747 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:06:29.237747 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:06:29.237747 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 21 18:06:29.237747 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:29.237747 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 404/817] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jun 21 18:06:29 2022] GET /identity/v3/services/network => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:29.238617 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedd2111-c208-4b15-a571-dffadcd05367 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:29.341953 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:29.345616 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedd2111-c208-4b15-a571-dffadcd05367 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:29.347752 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c462742-b341-46b4-b967-75b7afadcffb None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:29.384251 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f1fdf8-9715-4516-aa42-ab6347f12847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:29.385192 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f1fdf8-9715-4516-aa42-ab6347f12847 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:29.385584 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f1fdf8-9715-4516-aa42-ab6347f12847 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:29.385959 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f1fdf8-9715-4516-aa42-ab6347f12847 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:29.390262 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac21468-e1c3-4621-922c-2411feb6c43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:29.391293 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac21468-e1c3-4621-922c-2411feb6c43d None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:29.391773 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac21468-e1c3-4621-922c-2411feb6c43d None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:29.392250 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac21468-e1c3-4621-922c-2411feb6c43d None admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:29.393320 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac21468-e1c3-4621-922c-2411feb6c43d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:29.393825 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac21468-e1c3-4621-922c-2411feb6c43d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:29.395645 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac21468-e1c3-4621-922c-2411feb6c43d None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:29.417570 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 414/818] 10.222.0.23 () {56 vars in 1149 bytes} [Tue Jun 21 18:06:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:29.418844 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac21468-e1c3-4621-922c-2411feb6c43d None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:29.469805 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f1fdf8-9715-4516-aa42-ab6347f12847 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:29.471311 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f1fdf8-9715-4516-aa42-ab6347f12847 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:29.474658 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:29.540132 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac21468-e1c3-4621-922c-2411feb6c43d None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:29.583463 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8d4607-1d44-441f-bcbf-a37bd045afd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:29.584853 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:29.585266 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8d4607-1d44-441f-bcbf-a37bd045afd8 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:29.586408 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8d4607-1d44-441f-bcbf-a37bd045afd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:29.586984 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8d4607-1d44-441f-bcbf-a37bd045afd8 None admin] PATH_INFO: `/v3/services` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:29.587998 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8d4607-1d44-441f-bcbf-a37bd045afd8 None admin] RBAC: Adding query filter params (name=network) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:29.588879 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8d4607-1d44-441f-bcbf-a37bd045afd8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:29.588990 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 405/819] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:06:29 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 18:06:29.590054 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f1fdf8-9715-4516-aa42-ab6347f12847 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:29.595211 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8d4607-1d44-441f-bcbf-a37bd045afd8 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:29.601528 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 415/820] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Jun 21 18:06:29 2022] GET /identity/v3/services?name=network => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:29.602967 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8d4607-1d44-441f-bcbf-a37bd045afd8 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:29.712294 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f1fdf8-9715-4516-aa42-ab6347f12847 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:29.721938 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8d4607-1d44-441f-bcbf-a37bd045afd8 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:29.733921 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:29.755620 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e8c79f-2ed2-4778-b7fa-e70ce689d713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:29.757056 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e8c79f-2ed2-4778-b7fa-e70ce689d713 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:29.757578 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e8c79f-2ed2-4778-b7fa-e70ce689d713 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:29.758066 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e8c79f-2ed2-4778-b7fa-e70ce689d713 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:29.763664 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe6bc2-ef46-42b4-a383-90c7add19e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:29.764502 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe6bc2-ef46-42b4-a383-90c7add19e9d None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:29.764784 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe6bc2-ef46-42b4-a383-90c7add19e9d None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:29.765064 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe6bc2-ef46-42b4-a383-90c7add19e9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:29.772213 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e8c79f-2ed2-4778-b7fa-e70ce689d713 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:29.775027 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e8c79f-2ed2-4778-b7fa-e70ce689d713 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:29.778233 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fe6bc2-ef46-42b4-a383-90c7add19e9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:29.779883 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fe6bc2-ef46-42b4-a383-90c7add19e9d None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:29.796548 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-c3e8c79f-2ed2-4778-b7fa-e70ce689d713 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:29.796548 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:29.796548 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:29.796548 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:29.796548 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:29.796548 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:29.796548 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:29.796548 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:29.796548 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:29.796548 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:29.796548 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:29.796548 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:29.796548 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:06:29.796548 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:06:29.796548 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:06:29.797535 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:06:29.797535 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:29.797535 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:29.797535 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:29.797535 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:29.797535 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:29.797535 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:29.797535 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:29.797535 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:29.797535 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:29.797535 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:29.797535 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:29.797535 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:29.797535 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:29.797535 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:29.797535 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:29.797535 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:29.798448 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:06:29.798448 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:06:29.798448 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:29.798448 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:29.800148 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-d4fe6bc2-ef46-42b4-a383-90c7add19e9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:29.800148 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:29.800148 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:29.800148 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:29.800148 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:29.800148 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:29.800148 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:29.800148 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:29.800148 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:29.800148 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:29.800148 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:29.800148 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:29.800148 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:06:29.800148 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:06:29.800148 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:06:29.801066 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:06:29.801066 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:29.801066 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:29.801066 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:29.801066 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:29.801066 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:29.801066 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:29.801066 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:29.801066 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:29.801066 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:29.801066 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:29.801066 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:29.801066 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:29.801066 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:29.801066 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:29.801066 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:29.801066 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:29.801984 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:06:29.801984 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:06:29.801984 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:29.801984 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:29.801984 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 416/821] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:06:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:29.802275 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe6bc2-ef46-42b4-a383-90c7add19e9d None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:29.808614 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 406/822] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:06:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:29.809995 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e8c79f-2ed2-4778-b7fa-e70ce689d713 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:29.871315 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:29.924872 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe6bc2-ef46-42b4-a383-90c7add19e9d None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:29.928998 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e8c79f-2ed2-4778-b7fa-e70ce689d713 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:29.968605 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a963ac1f-397d-42ba-b442-46b79e13da87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:29.969372 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a963ac1f-397d-42ba-b442-46b79e13da87 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:29.969620 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a963ac1f-397d-42ba-b442-46b79e13da87 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:29.969901 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a963ac1f-397d-42ba-b442-46b79e13da87 None admin] PATH_INFO: `/v3/services` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:29.970716 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a963ac1f-397d-42ba-b442-46b79e13da87 None admin] RBAC: Adding query filter params (type=network) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:29.971025 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a963ac1f-397d-42ba-b442-46b79e13da87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:29.972803 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a963ac1f-397d-42ba-b442-46b79e13da87 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:29.975433 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e109de62-6c96-4b08-82dc-6fddbe7fa11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:29.976790 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109de62-6c96-4b08-82dc-6fddbe7fa11d None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:29.977293 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109de62-6c96-4b08-82dc-6fddbe7fa11d None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:29.978289 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109de62-6c96-4b08-82dc-6fddbe7fa11d None admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:29.979483 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e109de62-6c96-4b08-82dc-6fddbe7fa11d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:29.979755 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 417/823] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Jun 21 18:06:29 2022] GET /identity/v3/services?type=network => generated 350 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:29.980241 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e109de62-6c96-4b08-82dc-6fddbe7fa11d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:29.980642 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a963ac1f-397d-42ba-b442-46b79e13da87 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:29.982257 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e109de62-6c96-4b08-82dc-6fddbe7fa11d None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:30.002617 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 407/824] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:06:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:30.004182 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e109de62-6c96-4b08-82dc-6fddbe7fa11d None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:30.012201 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:30.104461 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a963ac1f-397d-42ba-b442-46b79e13da87 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:30.123997 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e109de62-6c96-4b08-82dc-6fddbe7fa11d None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:30.143314 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:30.148256 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdf18e7-bed4-492b-9b5a-80b99e89d112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:30.148986 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdf18e7-bed4-492b-9b5a-80b99e89d112 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:30.149274 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdf18e7-bed4-492b-9b5a-80b99e89d112 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:30.150251 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdf18e7-bed4-492b-9b5a-80b99e89d112 None admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:30.150879 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdf18e7-bed4-492b-9b5a-80b99e89d112 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:30.151192 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdf18e7-bed4-492b-9b5a-80b99e89d112 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:30.152702 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdf18e7-bed4-492b-9b5a-80b99e89d112 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:30.167048 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 418/825] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:06:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:30.169681 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdf18e7-bed4-492b-9b5a-80b99e89d112 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:30.187422 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637f8f66-0c46-4688-aaa6-2f5801d5c28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:30.188832 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f8f66-0c46-4688-aaa6-2f5801d5c28d None admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:30.189657 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f8f66-0c46-4688-aaa6-2f5801d5c28d None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:30.190248 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f8f66-0c46-4688-aaa6-2f5801d5c28d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:30.191815 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637f8f66-0c46-4688-aaa6-2f5801d5c28d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:30.194559 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637f8f66-0c46-4688-aaa6-2f5801d5c28d None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:30.267244 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:30.285282 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdf18e7-bed4-492b-9b5a-80b99e89d112 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:30.323496 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 408/826] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Jun 21 18:06:30 2022] POST /identity/v3/endpoints => generated 327 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:06:30.326346 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2b0e26-ec06-4168-a318-17520754a561 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:30.326865 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2b0e26-ec06-4168-a318-17520754a561 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:30.327361 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2b0e26-ec06-4168-a318-17520754a561 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:30.346301 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 409/827] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:06:30 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:06:30.346301 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2b0e26-ec06-4168-a318-17520754a561 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:30.364832 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1c8aff-a4ee-4f8e-9455-05417f1f0429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:30.366699 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c8aff-a4ee-4f8e-9455-05417f1f0429 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:30.367236 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c8aff-a4ee-4f8e-9455-05417f1f0429 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:30.367723 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c8aff-a4ee-4f8e-9455-05417f1f0429 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:30.379171 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1c8aff-a4ee-4f8e-9455-05417f1f0429 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:30.381427 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1c8aff-a4ee-4f8e-9455-05417f1f0429 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:30.401170 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-4d1c8aff-a4ee-4f8e-9455-05417f1f0429 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:06:30.401170 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:30.401170 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:30.401170 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:30.401170 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:30.401170 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:30.401170 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:30.401170 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:30.401170 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:30.401170 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:30.401170 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:30.401170 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:30.401170 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:06:30.401170 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:06:30.401170 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:06:30.402181 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:06:30.402181 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:30.402181 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:30.402181 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:30.402181 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:30.402181 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:30.402181 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:30.402181 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:30.402181 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:30.402181 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:30.402181 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:30.402181 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:30.402181 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:30.402181 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:30.402181 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:30.402181 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:30.402181 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:30.403095 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:06:30.403095 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:06:30.403095 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:06:30.403095 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:06:30.403095 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:06:30.403095 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:06:30.403095 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:06:30.403095 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:30.410908 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 419/828] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jun 21 18:06:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:30.411687 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ce85b-bc9d-4e9c-957e-e1faffe80608 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:30.411796 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ce85b-bc9d-4e9c-957e-e1faffe80608 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:30.412200 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ce85b-bc9d-4e9c-957e-e1faffe80608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:30.436555 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-4d4ce85b-bc9d-4e9c-957e-e1faffe80608 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:06:30.438197 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-4d4ce85b-bc9d-4e9c-957e-e1faffe80608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:06:30.536710 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 420/829] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:06:30 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:06:30.536710 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4ce85b-bc9d-4e9c-957e-e1faffe80608 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:30.585783 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2b0e26-ec06-4168-a318-17520754a561 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:30.665489 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:30.673831 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd1301-d9db-415d-97f1-6f7d58dcdba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:30.673831 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd1301-d9db-415d-97f1-6f7d58dcdba3 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:30.674110 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd1301-d9db-415d-97f1-6f7d58dcdba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:30.674455 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd1301-d9db-415d-97f1-6f7d58dcdba3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:30.693215 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fd1301-d9db-415d-97f1-6f7d58dcdba3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:30.697412 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fd1301-d9db-415d-97f1-6f7d58dcdba3 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:30.717571 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4ce85b-bc9d-4e9c-957e-e1faffe80608 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:30.725513 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-17fd1301-d9db-415d-97f1-6f7d58dcdba3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:06:30.725513 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:30.725513 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:30.725513 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:30.725513 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:30.725513 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:30.725513 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:30.725513 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:30.725513 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:30.725513 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:30.725513 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:30.725513 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:30.725513 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:06:30.725513 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:06:30.725513 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:06:30.727416 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:06:30.727416 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:30.727416 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:30.727416 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:30.727416 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:30.727416 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:30.727416 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:30.727416 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:30.727416 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:30.727416 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:30.727416 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:30.727416 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:30.727416 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:30.727416 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:30.727416 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:30.727416 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:30.727416 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:30.730001 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:06:30.730001 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:06:30.730001 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:06:30.730001 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:06:30.730001 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:06:30.730001 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:06:30.730001 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:06:30.730001 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:30.730001 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 410/830] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jun 21 18:06:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 385 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:30.730001 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd1301-d9db-415d-97f1-6f7d58dcdba3 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:30.769645 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abaa948-9cca-4d09-a0ed-938b5096d474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:30.770320 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abaa948-9cca-4d09-a0ed-938b5096d474 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:30.770596 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abaa948-9cca-4d09-a0ed-938b5096d474 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:30.770871 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abaa948-9cca-4d09-a0ed-938b5096d474 None admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:30.771676 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abaa948-9cca-4d09-a0ed-938b5096d474 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:30.771982 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abaa948-9cca-4d09-a0ed-938b5096d474 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:30.774112 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abaa948-9cca-4d09-a0ed-938b5096d474 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:30.784545 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:30.798223 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 421/831] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jun 21 18:06:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:30.798223 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abaa948-9cca-4d09-a0ed-938b5096d474 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:30.910467 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd1301-d9db-415d-97f1-6f7d58dcdba3 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:30.967516 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abaa948-9cca-4d09-a0ed-938b5096d474 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:30.974734 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e715a4a-a24c-4656-a68d-ac6924eb602d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:30.975970 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e715a4a-a24c-4656-a68d-ac6924eb602d None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:30.976873 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e715a4a-a24c-4656-a68d-ac6924eb602d None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:30.978697 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e715a4a-a24c-4656-a68d-ac6924eb602d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:31.004480 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:31.004743 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e715a4a-a24c-4656-a68d-ac6924eb602d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:06:31.011576 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171b0893-3740-4e5d-9fc3-12af1ae87713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:31.015589 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171b0893-3740-4e5d-9fc3-12af1ae87713 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:31.015589 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171b0893-3740-4e5d-9fc3-12af1ae87713 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:31.015589 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171b0893-3740-4e5d-9fc3-12af1ae87713 None admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:31.015589 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171b0893-3740-4e5d-9fc3-12af1ae87713 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:31.015589 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171b0893-3740-4e5d-9fc3-12af1ae87713 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:31.016285 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171b0893-3740-4e5d-9fc3-12af1ae87713 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:31.018942 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-4e715a4a-a24c-4656-a68d-ac6924eb602d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:06:31.018942 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:31.018942 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:31.018942 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:31.018942 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:31.018942 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:31.018942 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:31.018942 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:31.018942 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:31.018942 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:31.018942 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:31.018942 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:31.018942 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:06:31.018942 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:06:31.018942 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:06:31.018942 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:06:31.020968 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:06:31.020968 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:06:31.020968 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:06:31.020968 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:06:31.020968 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:31.020968 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:31.020968 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:31.020968 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:31.020968 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:31.020968 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:31.020968 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:31.020968 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:31.020968 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:31.020968 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:31.020968 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:31.020968 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:31.022632 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:31.022632 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:31.022632 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:31.022632 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:31.022632 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:06:31.022632 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:06:31.022632 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:06:31.022632 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:06:31.022632 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:06:31.022632 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:06:31.022632 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:06:31.022632 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:31.022632 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 411/832] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jun 21 18:06:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:31.026131 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e715a4a-a24c-4656-a68d-ac6924eb602d None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:31.046057 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 422/833] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jun 21 18:06:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:31.047446 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171b0893-3740-4e5d-9fc3-12af1ae87713 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:31.165747 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:31.182667 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e715a4a-a24c-4656-a68d-ac6924eb602d None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:31.229252 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25d59f0-bd62-4231-b9ab-3304c06f9694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:31.230275 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25d59f0-bd62-4231-b9ab-3304c06f9694 None admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:31.230560 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25d59f0-bd62-4231-b9ab-3304c06f9694 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:31.230833 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25d59f0-bd62-4231-b9ab-3304c06f9694 None admin] PATH_INFO: `/v3/projects/2a4699d1ec70456da5f8e0aa4b1439b7/users/fcfcd9f134e74b09bfdac337c81c8a07/roles/acf4e26e0ceb41ddad002dd4edf6373c` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:31.255521 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25d59f0-bd62-4231-b9ab-3304c06f9694 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a4699d1ec70456da5f8e0aa4b1439b7, user_id=fcfcd9f134e74b09bfdac337c81c8a07, role_id=acf4e26e0ceb41ddad002dd4edf6373c)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:31.257059 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25d59f0-bd62-4231-b9ab-3304c06f9694 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:31.325508 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 412/834] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jun 21 18:06:31 2022] PUT /identity/v3/projects/2a4699d1ec70456da5f8e0aa4b1439b7/users/fcfcd9f134e74b09bfdac337c81c8a07/roles/acf4e26e0ceb41ddad002dd4edf6373c => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:06:31.326920 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25d59f0-bd62-4231-b9ab-3304c06f9694 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:31.372203 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171b0893-3740-4e5d-9fc3-12af1ae87713 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:31.417524 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541478ea-4aba-43ba-8d93-cae613712a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:31.418242 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541478ea-4aba-43ba-8d93-cae613712a12 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:31.418419 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541478ea-4aba-43ba-8d93-cae613712a12 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:31.418692 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541478ea-4aba-43ba-8d93-cae613712a12 None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:31.419472 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541478ea-4aba-43ba-8d93-cae613712a12 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:31.419778 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541478ea-4aba-43ba-8d93-cae613712a12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:31.421246 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541478ea-4aba-43ba-8d93-cae613712a12 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:31.432968 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 423/835] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:06:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:31.434492 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad459efb-51e7-46f5-b3bc-2c33bb8e91a2 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:31.434818 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad459efb-51e7-46f5-b3bc-2c33bb8e91a2 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:31.435065 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad459efb-51e7-46f5-b3bc-2c33bb8e91a2 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:31.436691 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 424/836] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:06:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:06:31.439098 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad459efb-51e7-46f5-b3bc-2c33bb8e91a2 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:31.480790 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25d59f0-bd62-4231-b9ab-3304c06f9694 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:31.517620 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:31.541845 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5d19a6-65ae-453b-aae0-94896a63e452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:31.542534 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5d19a6-65ae-453b-aae0-94896a63e452 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:31.542862 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5d19a6-65ae-453b-aae0-94896a63e452 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:31.543093 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5d19a6-65ae-453b-aae0-94896a63e452 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:31.544424 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5d19a6-65ae-453b-aae0-94896a63e452 None admin] RBAC: Adding query filter params (role.id=745ed9065a8143e384ebbd694b29635c, scope.project.id=2a4699d1ec70456da5f8e0aa4b1439b7, user.id=904255dd3c1f406eb10f6fc65f9cd098) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:31.544424 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5d19a6-65ae-453b-aae0-94896a63e452 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:31.545787 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5d19a6-65ae-453b-aae0-94896a63e452 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:31.552992 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 413/837] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jun 21 18:06:31 2022] GET /identity/v3/role_assignments?user.id=904255dd3c1f406eb10f6fc65f9cd098&scope.project.id=2a4699d1ec70456da5f8e0aa4b1439b7&role.id=745ed9065a8143e384ebbd694b29635c => generated 608 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:31.554151 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854b3c1-ed3c-4de5-92c9-fbb0b8c86c14 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:31.554440 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854b3c1-ed3c-4de5-92c9-fbb0b8c86c14 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:31.554747 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854b3c1-ed3c-4de5-92c9-fbb0b8c86c14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:31.560461 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:31.574165 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-b854b3c1-ed3c-4de5-92c9-fbb0b8c86c14 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:06:31.575364 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-b854b3c1-ed3c-4de5-92c9-fbb0b8c86c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:06:31.586893 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad459efb-51e7-46f5-b3bc-2c33bb8e91a2 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:31.636442 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea65232-fe30-4269-b85d-0faa696d629d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:31.636442 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea65232-fe30-4269-b85d-0faa696d629d None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:31.636442 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea65232-fe30-4269-b85d-0faa696d629d None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:31.636442 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea65232-fe30-4269-b85d-0faa696d629d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:31.655573 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea65232-fe30-4269-b85d-0faa696d629d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:31.657393 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea65232-fe30-4269-b85d-0faa696d629d None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:31.685963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-fea65232-fe30-4269-b85d-0faa696d629d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:31.685963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:31.685963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:31.685963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:31.685963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:31.685963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:31.685963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:31.685963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:31.685963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:31.685963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:31.685963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:31.685963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:31.685963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:06:31.685963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:06:31.685963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:06:31.686902 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:06:31.686902 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:31.686902 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:31.686902 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:31.686902 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:31.686902 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:31.686902 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:31.686902 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:31.686902 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:31.686902 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:31.686902 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:31.686902 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:31.686902 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:31.686902 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:31.686902 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:31.686902 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:31.686902 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:31.687833 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:06:31.687833 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:06:31.687833 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:31.687833 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:31.687833 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 425/838] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:06:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:31.691806 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 414/839] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:06:31 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:06:31.697226 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea65232-fe30-4269-b85d-0faa696d629d None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:31.717387 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b854b3c1-ed3c-4de5-92c9-fbb0b8c86c14 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:31.841936 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea65232-fe30-4269-b85d-0faa696d629d None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:31.845007 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:31.857044 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b854b3c1-ed3c-4de5-92c9-fbb0b8c86c14 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:31.886924 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fe1f21-e352-4b79-bd9b-7d90fb67e8d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:31.888051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fe1f21-e352-4b79-bd9b-7d90fb67e8d1 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:31.888346 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fe1f21-e352-4b79-bd9b-7d90fb67e8d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:31.888684 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fe1f21-e352-4b79-bd9b-7d90fb67e8d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:31.889422 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fe1f21-e352-4b79-bd9b-7d90fb67e8d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:31.889736 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fe1f21-e352-4b79-bd9b-7d90fb67e8d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:31.891451 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fe1f21-e352-4b79-bd9b-7d90fb67e8d1 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:31.897421 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:31.904096 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeb94c1-1510-44e1-a527-bf755a35b3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:31.906812 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb94c1-1510-44e1-a527-bf755a35b3dd None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:31.906812 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb94c1-1510-44e1-a527-bf755a35b3dd None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:31.906812 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb94c1-1510-44e1-a527-bf755a35b3dd None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:31.906812 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeb94c1-1510-44e1-a527-bf755a35b3dd None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:31.907133 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 426/840] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:06:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:31.907833 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeb94c1-1510-44e1-a527-bf755a35b3dd None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:31.912334 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fe1f21-e352-4b79-bd9b-7d90fb67e8d1 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:31.932374 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-afeb94c1-1510-44e1-a527-bf755a35b3dd None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 21 18:06:31.932374 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:31.932374 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:06:31.932374 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:06:31.932374 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:06:31.932374 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:06:31.932374 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:06:31.932374 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:06:31.932374 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 21 18:06:31.932374 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:31.932374 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:06:31.932374 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:31.932374 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:31.932374 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:31.932374 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:31.932374 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:31.932374 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:31.934214 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:31.934214 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:31.934214 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:31.934214 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:31.934214 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:31.934214 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:31.934214 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:06:31.934214 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:06:31.934214 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:06:31.934214 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:06:31.934214 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:31.934214 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:31.934214 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:31.934214 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:31.934214 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:31.934214 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:31.936905 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:31.936905 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:31.936905 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:31.936905 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:31.936905 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:31.936905 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:31.936905 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:31.936905 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:31.936905 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:31.936905 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:31.936905 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:06:31.936905 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:06:31.936905 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 21 18:06:31.936905 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:31.936905 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 415/841] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jun 21 18:06:31 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:06:31.945686 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeb94c1-1510-44e1-a527-bf755a35b3dd None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:32.077232 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fe1f21-e352-4b79-bd9b-7d90fb67e8d1 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:32.117728 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f62e2c-625e-4522-8722-ddc353ea4e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:32.119153 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f62e2c-625e-4522-8722-ddc353ea4e0d None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:32.119462 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f62e2c-625e-4522-8722-ddc353ea4e0d None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:32.119714 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f62e2c-625e-4522-8722-ddc353ea4e0d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:32.121195 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeb94c1-1510-44e1-a527-bf755a35b3dd None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:32.130228 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f62e2c-625e-4522-8722-ddc353ea4e0d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:32.131740 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f62e2c-625e-4522-8722-ddc353ea4e0d None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:32.150486 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-20f62e2c-625e-4522-8722-ddc353ea4e0d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 21 18:06:32.150486 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:32.150486 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:32.150486 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:32.150486 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:32.150486 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:32.150486 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:32.150486 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:32.150486 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:32.150486 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:32.150486 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:32.150486 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:32.150486 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:06:32.150486 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:06:32.150486 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:06:32.150486 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:06:32.152303 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:32.152303 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:32.152303 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:06:32.152303 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:06:32.152303 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:06:32.152303 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:06:32.152303 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:32.152303 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:32.152303 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:32.152303 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:32.152303 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:32.152303 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:32.152303 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:32.152303 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:32.152303 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:32.152303 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:32.152303 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:32.152303 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:32.154668 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:32.154668 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:32.154668 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:06:32.154668 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:06:32.154668 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:06:32.154668 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:06:32.154668 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:06:32.154668 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:06:32.154668 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 21 18:06:32.154668 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:32.156846 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 427/842] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jun 21 18:06:31 2022] GET /identity/v3/users/swift => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:32.157857 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f62e2c-625e-4522-8722-ddc353ea4e0d None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:32.188245 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f0571c-6e08-416c-a2c9-94dc3bbd3cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:32.188935 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f0571c-6e08-416c-a2c9-94dc3bbd3cf0 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:32.189229 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f0571c-6e08-416c-a2c9-94dc3bbd3cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:32.189709 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f0571c-6e08-416c-a2c9-94dc3bbd3cf0 None admin] PATH_INFO: `/v3/services` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:32.190411 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f0571c-6e08-416c-a2c9-94dc3bbd3cf0 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:32.190724 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f0571c-6e08-416c-a2c9-94dc3bbd3cf0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:32.192123 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f0571c-6e08-416c-a2c9-94dc3bbd3cf0 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:32.204097 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 416/843] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jun 21 18:06:31 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:32.204097 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f0571c-6e08-416c-a2c9-94dc3bbd3cf0 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:32.208293 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:32.297189 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:32.317952 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f62e2c-625e-4522-8722-ddc353ea4e0d None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:32.350294 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f0571c-6e08-416c-a2c9-94dc3bbd3cf0 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:32.369694 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddab363d-7ae3-40a5-9c7f-4e162a79f172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:32.370900 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddab363d-7ae3-40a5-9c7f-4e162a79f172 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:32.371377 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddab363d-7ae3-40a5-9c7f-4e162a79f172 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:32.372072 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddab363d-7ae3-40a5-9c7f-4e162a79f172 None admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:32.373554 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddab363d-7ae3-40a5-9c7f-4e162a79f172 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:32.374474 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddab363d-7ae3-40a5-9c7f-4e162a79f172 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:32.376592 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddab363d-7ae3-40a5-9c7f-4e162a79f172 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:32.394845 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e08f266-2e26-41b0-b78a-13cecdb0cb47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:32.395526 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e08f266-2e26-41b0-b78a-13cecdb0cb47 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:32.395813 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e08f266-2e26-41b0-b78a-13cecdb0cb47 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:32.396094 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e08f266-2e26-41b0-b78a-13cecdb0cb47 None admin] PATH_INFO: `/v3/services` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:32.396860 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e08f266-2e26-41b0-b78a-13cecdb0cb47 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:32.397178 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e08f266-2e26-41b0-b78a-13cecdb0cb47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:32.398576 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e08f266-2e26-41b0-b78a-13cecdb0cb47 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:32.405077 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 428/844] 10.222.0.23 () {56 vars in 1135 bytes} [Tue Jun 21 18:06:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:32.416462 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 417/845] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jun 21 18:06:32 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:32.416720 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddab363d-7ae3-40a5-9c7f-4e162a79f172 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:32.512798 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:32.534789 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddab363d-7ae3-40a5-9c7f-4e162a79f172 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:32.576874 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec926f-8c4b-4513-9239-6eeb75ac6b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:32.577549 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec926f-8c4b-4513-9239-6eeb75ac6b6d None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:32.577830 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec926f-8c4b-4513-9239-6eeb75ac6b6d None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:32.578104 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec926f-8c4b-4513-9239-6eeb75ac6b6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:32.661068 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ec926f-8c4b-4513-9239-6eeb75ac6b6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:32.662878 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ec926f-8c4b-4513-9239-6eeb75ac6b6d None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:32.755119 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:32.777173 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 429/846] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:06:32 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 18:06:32.782765 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e08f266-2e26-41b0-b78a-13cecdb0cb47 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:32.895605 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e08f266-2e26-41b0-b78a-13cecdb0cb47 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:32.933750 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f54f106-3224-4cf8-891c-6429b3beb3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:32.934596 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f54f106-3224-4cf8-891c-6429b3beb3e6 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:32.934929 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f54f106-3224-4cf8-891c-6429b3beb3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:32.935214 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f54f106-3224-4cf8-891c-6429b3beb3e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:32.938448 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:32.948187 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f54f106-3224-4cf8-891c-6429b3beb3e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:32.949765 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f54f106-3224-4cf8-891c-6429b3beb3e6 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:32.968257 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-0f54f106-3224-4cf8-891c-6429b3beb3e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:32.968257 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:32.968257 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:32.968257 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:32.968257 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:32.968257 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:32.968257 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:32.968257 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:32.968257 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:32.968257 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:32.968257 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:32.968257 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:32.968257 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:06:32.968257 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:06:32.968257 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:06:32.969384 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:06:32.969384 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:32.969384 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:32.969384 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:32.969384 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:32.969384 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:32.969384 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:32.969384 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:32.969384 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:32.969384 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:32.969384 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:32.969384 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:32.969384 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:32.969384 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:32.969384 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:32.969384 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:32.969384 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:32.970301 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:06:32.970301 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:06:32.970301 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:32.970301 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:32.970301 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 418/847] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:06:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:06:32.975673 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec926f-8c4b-4513-9239-6eeb75ac6b6d None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:33.087495 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec926f-8c4b-4513-9239-6eeb75ac6b6d None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:33.125535 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b46795-9b61-467c-a91a-ccae40b874dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:33.126235 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b46795-9b61-467c-a91a-ccae40b874dc None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:33.126519 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b46795-9b61-467c-a91a-ccae40b874dc None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:33.126790 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b46795-9b61-467c-a91a-ccae40b874dc None admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:33.127544 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b46795-9b61-467c-a91a-ccae40b874dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:33.127850 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b46795-9b61-467c-a91a-ccae40b874dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:33.129297 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b46795-9b61-467c-a91a-ccae40b874dc None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:33.142929 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 430/848] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:06:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:33.148383 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f54f106-3224-4cf8-891c-6429b3beb3e6 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:33.263492 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f54f106-3224-4cf8-891c-6429b3beb3e6 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:33.299170 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc1f547-e69e-4d07-845c-3af18927fc2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:33.299967 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc1f547-e69e-4d07-845c-3af18927fc2a None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:33.300247 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc1f547-e69e-4d07-845c-3af18927fc2a None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:33.300548 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc1f547-e69e-4d07-845c-3af18927fc2a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:33.308736 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc1f547-e69e-4d07-845c-3af18927fc2a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:33.310197 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc1f547-e69e-4d07-845c-3af18927fc2a None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:33.326582 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-0cc1f547-e69e-4d07-845c-3af18927fc2a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:06:33.326582 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:33.326582 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:33.326582 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:33.326582 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:33.326582 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:33.326582 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:33.326582 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:33.326582 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:33.326582 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:33.326582 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:33.326582 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:33.326582 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:06:33.326582 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:06:33.326582 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:06:33.327793 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:06:33.327793 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:33.327793 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:33.327793 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:33.327793 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:33.327793 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:33.327793 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:33.327793 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:33.327793 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:33.327793 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:33.327793 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:33.327793 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:33.327793 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:33.327793 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:33.327793 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:33.327793 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:33.327793 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:33.328764 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:06:33.328764 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:06:33.328764 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:06:33.328764 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:06:33.328764 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:06:33.328764 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:06:33.328764 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:06:33.328764 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:33.328764 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 419/849] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jun 21 18:06:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:33.333888 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b46795-9b61-467c-a91a-ccae40b874dc None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:33.384266 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:33.446726 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b46795-9b61-467c-a91a-ccae40b874dc None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:33.483659 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a4a12c-851c-42aa-9384-7aad06f455eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:33.484334 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a4a12c-851c-42aa-9384-7aad06f455eb None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:33.484674 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a4a12c-851c-42aa-9384-7aad06f455eb None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:33.484948 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a4a12c-851c-42aa-9384-7aad06f455eb None admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:33.485741 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a4a12c-851c-42aa-9384-7aad06f455eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:33.486077 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a4a12c-851c-42aa-9384-7aad06f455eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:33.487514 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a4a12c-851c-42aa-9384-7aad06f455eb None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:33.502696 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 431/850] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jun 21 18:06:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:33.508567 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc1f547-e69e-4d07-845c-3af18927fc2a None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:33.550595 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:33.593853 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11bee12-030d-4dd0-a2ca-66c5a49d11d6 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:33.594135 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11bee12-030d-4dd0-a2ca-66c5a49d11d6 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:33.594412 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11bee12-030d-4dd0-a2ca-66c5a49d11d6 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:33.596018 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 432/851] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:06:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:06:33.601956 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd4124-70dd-415b-8e7e-874fcda5afa8 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:33.602362 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd4124-70dd-415b-8e7e-874fcda5afa8 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:33.603941 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd4124-70dd-415b-8e7e-874fcda5afa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:33.621875 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-f9cd4124-70dd-415b-8e7e-874fcda5afa8 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:06:33.623241 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-f9cd4124-70dd-415b-8e7e-874fcda5afa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:06:33.627389 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc1f547-e69e-4d07-845c-3af18927fc2a None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:33.666564 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b9bae6-b3ed-4b84-be46-ca924126766d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:33.667422 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b9bae6-b3ed-4b84-be46-ca924126766d None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:33.667781 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b9bae6-b3ed-4b84-be46-ca924126766d None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:33.668698 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b9bae6-b3ed-4b84-be46-ca924126766d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:33.669862 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b9bae6-b3ed-4b84-be46-ca924126766d None admin] RBAC: Adding query filter params (role.id=745ed9065a8143e384ebbd694b29635c, scope.project.id=2a4699d1ec70456da5f8e0aa4b1439b7, user.id=0e0d25565f1543d9a7577288fc09119f) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:33.670285 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b9bae6-b3ed-4b84-be46-ca924126766d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:33.672198 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b9bae6-b3ed-4b84-be46-ca924126766d None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:33.680760 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 420/852] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jun 21 18:06:33 2022] GET /identity/v3/role_assignments?user.id=0e0d25565f1543d9a7577288fc09119f&scope.project.id=2a4699d1ec70456da5f8e0aa4b1439b7&role.id=745ed9065a8143e384ebbd694b29635c => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:33.700951 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 433/853] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:06:33 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:06:33.726579 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b9bae6-b3ed-4b84-be46-ca924126766d None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:33.838687 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7eb3f3-dc96-4965-acec-001dad966733 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:33.839060 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7eb3f3-dc96-4965-acec-001dad966733 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:33.839284 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7eb3f3-dc96-4965-acec-001dad966733 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:33.844894 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 434/854] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:06:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:06:33.846930 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d667ebf-eba6-472d-8745-a4ec9ad29dc2 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:33.847287 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d667ebf-eba6-472d-8745-a4ec9ad29dc2 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:33.847566 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d667ebf-eba6-472d-8745-a4ec9ad29dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:33.850691 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:33.854315 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b9bae6-b3ed-4b84-be46-ca924126766d None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:33.872690 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-5d667ebf-eba6-472d-8745-a4ec9ad29dc2 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:06:33.873951 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-5d667ebf-eba6-472d-8745-a4ec9ad29dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:06:33.898188 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1bdee1-8e4b-46b7-aa61-b72665a264c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:33.898524 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1bdee1-8e4b-46b7-aa61-b72665a264c0 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:33.898732 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1bdee1-8e4b-46b7-aa61-b72665a264c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:33.899012 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1bdee1-8e4b-46b7-aa61-b72665a264c0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:33.911279 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1bdee1-8e4b-46b7-aa61-b72665a264c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:06:33.919788 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:33.920229 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-fd1bdee1-8e4b-46b7-aa61-b72665a264c0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:06:33.920229 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:33.920229 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:33.920229 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:33.920229 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:33.920229 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:33.920229 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:33.920229 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:33.920229 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:33.920229 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:33.920229 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:33.920229 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:33.920229 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:06:33.920229 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:06:33.920229 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:06:33.920229 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:06:33.921286 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:06:33.921286 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:06:33.921286 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:06:33.921286 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:06:33.921286 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:33.921286 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:33.921286 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:33.921286 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:33.921286 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:33.921286 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:33.921286 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:33.921286 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:33.921286 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:33.921286 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:33.921286 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:33.921286 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:33.922642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:33.922642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:33.922642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:33.922642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:33.922642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:06:33.922642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:06:33.922642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:06:33.922642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:06:33.922642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:06:33.922642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:06:33.922642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:06:33.922642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:33.922642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 421/855] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jun 21 18:06:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:33.927429 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1bdee1-8e4b-46b7-aa61-b72665a264c0 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:33.976192 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 435/856] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:06:33 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:06:33.999377 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d667ebf-eba6-472d-8745-a4ec9ad29dc2 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:34.048931 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1bdee1-8e4b-46b7-aa61-b72665a264c0 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:34.085917 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69e140c-df54-464b-bf26-79daf0982664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:34.087483 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e140c-df54-464b-bf26-79daf0982664 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:34.087942 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e140c-df54-464b-bf26-79daf0982664 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:34.088427 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e140c-df54-464b-bf26-79daf0982664 None admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:34.089351 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69e140c-df54-464b-bf26-79daf0982664 None admin] RBAC: Adding query filter params (name=service) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:34.089827 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69e140c-df54-464b-bf26-79daf0982664 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:34.091570 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69e140c-df54-464b-bf26-79daf0982664 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:34.101744 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 422/857] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jun 21 18:06:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:06:34.108545 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69e140c-df54-464b-bf26-79daf0982664 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:34.134457 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d667ebf-eba6-472d-8745-a4ec9ad29dc2 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:34.136531 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:34.187372 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a13b93f-ca78-4839-bc76-5a7d182f9c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:34.188331 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13b93f-ca78-4839-bc76-5a7d182f9c51 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:34.188814 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13b93f-ca78-4839-bc76-5a7d182f9c51 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:34.189162 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13b93f-ca78-4839-bc76-5a7d182f9c51 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:34.189993 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a13b93f-ca78-4839-bc76-5a7d182f9c51 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:34.191933 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a13b93f-ca78-4839-bc76-5a7d182f9c51 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:34.206284 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-2a13b93f-ca78-4839-bc76-5a7d182f9c51 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 21 18:06:34.206284 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:34.206284 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:06:34.206284 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:06:34.206284 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:06:34.206284 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:06:34.206284 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:06:34.206284 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:06:34.206284 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 21 18:06:34.206284 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:34.206284 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:06:34.206284 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:34.206284 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:34.206284 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:34.206284 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:34.206284 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:34.207638 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:34.207638 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:34.207638 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:34.207638 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:34.207638 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:34.207638 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:34.207638 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:34.207638 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:06:34.207638 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:06:34.207638 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:06:34.207638 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:06:34.207638 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:34.207638 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:34.207638 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:34.207638 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:34.207638 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:34.208818 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:34.208818 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:34.208818 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:34.208818 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:34.208818 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:34.208818 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:34.208818 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:34.208818 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:34.208818 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:34.208818 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:34.208818 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:34.208818 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:06:34.208818 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:06:34.208818 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 21 18:06:34.208818 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:34.216054 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 436/858] 10.222.0.23 () {56 vars in 1132 bytes} [Tue Jun 21 18:06:33 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:06:34.217714 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a13b93f-ca78-4839-bc76-5a7d182f9c51 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:34.241043 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69e140c-df54-464b-bf26-79daf0982664 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:34.258828 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:34.290807 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809cec19-21b0-405f-b3b2-a82f4e8519a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:34.292662 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809cec19-21b0-405f-b3b2-a82f4e8519a7 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:34.293135 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809cec19-21b0-405f-b3b2-a82f4e8519a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:34.293496 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809cec19-21b0-405f-b3b2-a82f4e8519a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:34.311938 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809cec19-21b0-405f-b3b2-a82f4e8519a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:34.313492 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809cec19-21b0-405f-b3b2-a82f4e8519a7 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:34.344809 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-809cec19-21b0-405f-b3b2-a82f4e8519a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:34.344809 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:34.344809 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:34.344809 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:34.344809 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:34.344809 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:34.344809 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:34.344809 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:34.344809 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:34.344809 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:34.344809 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:34.344809 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:34.344809 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:06:34.344809 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:06:34.344809 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:06:34.346224 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:06:34.346224 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:34.346224 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:34.346224 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:34.346224 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:34.346224 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:34.346224 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:34.346224 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:34.346224 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:34.346224 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:34.346224 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:34.346224 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:34.346224 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:34.346224 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:34.346224 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:34.346224 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:34.346224 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:34.347397 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:06:34.347397 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:06:34.347397 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:34.347397 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:34.356224 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 423/859] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:06:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:34.357918 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809cec19-21b0-405f-b3b2-a82f4e8519a7 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:34.364921 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a13b93f-ca78-4839-bc76-5a7d182f9c51 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:34.403699 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17037946-e886-4e3c-bae9-93742893a26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:34.404377 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17037946-e886-4e3c-bae9-93742893a26d None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:34.404696 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17037946-e886-4e3c-bae9-93742893a26d None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:34.404981 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17037946-e886-4e3c-bae9-93742893a26d None admin] PATH_INFO: `/v3/services` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:34.405771 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17037946-e886-4e3c-bae9-93742893a26d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:34.406580 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17037946-e886-4e3c-bae9-93742893a26d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:34.407495 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17037946-e886-4e3c-bae9-93742893a26d None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:34.418082 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 437/860] 10.222.0.23 () {56 vars in 1126 bytes} [Tue Jun 21 18:06:34 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:34.419529 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17037946-e886-4e3c-bae9-93742893a26d None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:34.447955 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:34.477023 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809cec19-21b0-405f-b3b2-a82f4e8519a7 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:34.520071 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232643fe-edfb-497d-87c9-a36fcb2e8da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:34.521107 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232643fe-edfb-497d-87c9-a36fcb2e8da1 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:34.521494 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232643fe-edfb-497d-87c9-a36fcb2e8da1 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:34.521953 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232643fe-edfb-497d-87c9-a36fcb2e8da1 None admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:34.523148 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232643fe-edfb-497d-87c9-a36fcb2e8da1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:34.523665 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232643fe-edfb-497d-87c9-a36fcb2e8da1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:34.525033 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:34.527051 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232643fe-edfb-497d-87c9-a36fcb2e8da1 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:34.536170 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17037946-e886-4e3c-bae9-93742893a26d None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:34.549672 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 424/861] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:06:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:34.550917 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232643fe-edfb-497d-87c9-a36fcb2e8da1 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:34.575891 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e0110-afe7-4844-a3ac-791903fb8005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:34.576911 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e0110-afe7-4844-a3ac-791903fb8005 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:34.577355 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e0110-afe7-4844-a3ac-791903fb8005 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:34.577776 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e0110-afe7-4844-a3ac-791903fb8005 None admin] PATH_INFO: `/v3/services` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:34.578757 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767e0110-afe7-4844-a3ac-791903fb8005 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:34.579318 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767e0110-afe7-4844-a3ac-791903fb8005 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:34.581023 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767e0110-afe7-4844-a3ac-791903fb8005 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:34.592546 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 438/862] 10.222.0.23 () {56 vars in 1126 bytes} [Tue Jun 21 18:06:34 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:34.679742 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232643fe-edfb-497d-87c9-a36fcb2e8da1 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:34.721082 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f773b9-3899-4c71-87e2-43decda016b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:34.722392 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f773b9-3899-4c71-87e2-43decda016b8 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:34.722710 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f773b9-3899-4c71-87e2-43decda016b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:34.722993 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f773b9-3899-4c71-87e2-43decda016b8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:34.725852 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:34.732744 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f773b9-3899-4c71-87e2-43decda016b8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:34.734641 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f773b9-3899-4c71-87e2-43decda016b8 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:34.751726 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-54f773b9-3899-4c71-87e2-43decda016b8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 18:06:34.751726 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:34.751726 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:34.751726 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:34.751726 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:34.751726 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:34.751726 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:34.751726 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:34.751726 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:34.751726 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:34.751726 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:34.751726 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:34.751726 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:06:34.751726 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:06:34.751726 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:06:34.751726 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:06:34.752817 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:34.752817 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:34.752817 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:06:34.752817 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:06:34.752817 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:06:34.752817 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:06:34.752817 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:34.752817 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:34.752817 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:34.752817 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:34.752817 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:34.752817 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:34.752817 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:34.752817 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:34.752817 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:34.752817 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:34.752817 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:34.752817 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:34.753811 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:34.753811 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:34.753811 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:06:34.753811 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:06:34.753811 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:06:34.753811 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:06:34.753811 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:06:34.753811 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:06:34.753811 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 18:06:34.753811 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:34.754396 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f2f553-bcef-44f0-b291-1e46293e4843 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:34.754396 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f2f553-bcef-44f0-b291-1e46293e4843 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:34.754396 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f2f553-bcef-44f0-b291-1e46293e4843 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:34.754593 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 425/863] 10.222.0.23 () {56 vars in 1117 bytes} [Tue Jun 21 18:06:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:06:34.754681 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 439/864] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:06:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:06:34.759160 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f773b9-3899-4c71-87e2-43decda016b8 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:34.762098 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa1e70-41a1-4c03-a6ab-c5c14385528c None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:34.762686 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa1e70-41a1-4c03-a6ab-c5c14385528c None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:34.763167 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa1e70-41a1-4c03-a6ab-c5c14385528c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:34.793576 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-fffa1e70-41a1-4c03-a6ab-c5c14385528c None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:06:34.795715 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-fffa1e70-41a1-4c03-a6ab-c5c14385528c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:06:34.905584 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 440/865] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:06:34 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:06:34.908777 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f773b9-3899-4c71-87e2-43decda016b8 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:34.929172 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffa1e70-41a1-4c03-a6ab-c5c14385528c None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:34.951138 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1272a94a-6158-4718-af43-b83de7d19ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:34.952087 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1272a94a-6158-4718-af43-b83de7d19ef3 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:34.952576 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1272a94a-6158-4718-af43-b83de7d19ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:34.953171 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1272a94a-6158-4718-af43-b83de7d19ef3 None admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:34.954204 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1272a94a-6158-4718-af43-b83de7d19ef3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:34.954684 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1272a94a-6158-4718-af43-b83de7d19ef3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:34.956438 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1272a94a-6158-4718-af43-b83de7d19ef3 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:34.979486 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 426/866] 10.222.0.23 () {56 vars in 1149 bytes} [Tue Jun 21 18:06:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:34.985893 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1272a94a-6158-4718-af43-b83de7d19ef3 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:35.024215 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:35.044766 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffa1e70-41a1-4c03-a6ab-c5c14385528c None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:35.052472 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:35.083773 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d749d8d3-7c88-4381-a548-e08914bc9af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:35.084700 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d749d8d3-7c88-4381-a548-e08914bc9af4 None admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:35.085082 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d749d8d3-7c88-4381-a548-e08914bc9af4 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:35.085880 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d749d8d3-7c88-4381-a548-e08914bc9af4 None admin] PATH_INFO: `/v3/services` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:35.086419 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d749d8d3-7c88-4381-a548-e08914bc9af4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:35.088934 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d749d8d3-7c88-4381-a548-e08914bc9af4 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:35.113655 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1272a94a-6158-4718-af43-b83de7d19ef3 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:35.153948 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fba552e-c6f6-4a9f-84f1-0e5670661179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:35.154620 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fba552e-c6f6-4a9f-84f1-0e5670661179 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:35.154908 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fba552e-c6f6-4a9f-84f1-0e5670661179 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:35.155419 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fba552e-c6f6-4a9f-84f1-0e5670661179 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:35.181718 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 441/867] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Jun 21 18:06:34 2022] POST /identity/v3/services => generated 251 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:06:35.259231 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fba552e-c6f6-4a9f-84f1-0e5670661179 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:35.261647 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fba552e-c6f6-4a9f-84f1-0e5670661179 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:35.322933 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:35.380550 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 427/868] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:06:34 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 21 18:06:35.390504 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d749d8d3-7c88-4381-a548-e08914bc9af4 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:35.536518 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d749d8d3-7c88-4381-a548-e08914bc9af4 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:35.577326 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83929696-6a45-4ae3-af35-b377387e7c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:35.579481 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83929696-6a45-4ae3-af35-b377387e7c4c None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:35.579991 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83929696-6a45-4ae3-af35-b377387e7c4c None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:35.580540 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83929696-6a45-4ae3-af35-b377387e7c4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:35.593527 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:35.597252 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83929696-6a45-4ae3-af35-b377387e7c4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:35.599686 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83929696-6a45-4ae3-af35-b377387e7c4c None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:35.624333 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-83929696-6a45-4ae3-af35-b377387e7c4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:35.624333 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:35.624333 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:35.624333 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:35.624333 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:35.624333 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:35.624333 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:35.624333 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:35.624333 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:35.624333 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:35.624333 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:35.624333 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:35.624333 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:06:35.624333 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:06:35.624333 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:06:35.625593 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:06:35.625593 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:35.625593 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:35.625593 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:35.625593 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:35.625593 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:35.625593 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:35.625593 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:35.625593 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:35.625593 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:35.625593 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:35.625593 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:35.625593 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:35.625593 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:35.625593 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:35.625593 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:35.625593 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:35.626536 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:06:35.626536 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:06:35.626536 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:35.626536 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:35.629133 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 442/869] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:06:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:06:35.635592 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fba552e-c6f6-4a9f-84f1-0e5670661179 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:35.764501 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fba552e-c6f6-4a9f-84f1-0e5670661179 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:35.806969 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4327853-6f9d-41c8-b4e7-5364d7d26c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:35.807426 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4327853-6f9d-41c8-b4e7-5364d7d26c50 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:35.807793 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4327853-6f9d-41c8-b4e7-5364d7d26c50 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:35.808073 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4327853-6f9d-41c8-b4e7-5364d7d26c50 None admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:35.808834 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4327853-6f9d-41c8-b4e7-5364d7d26c50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:35.809143 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4327853-6f9d-41c8-b4e7-5364d7d26c50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:35.810522 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4327853-6f9d-41c8-b4e7-5364d7d26c50 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:35.828604 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 428/870] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:06:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:35.835662 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83929696-6a45-4ae3-af35-b377387e7c4c None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:35.969212 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83929696-6a45-4ae3-af35-b377387e7c4c None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:36.005554 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0cc43e-2db9-4454-94c4-036f6caeed5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:36.006513 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0cc43e-2db9-4454-94c4-036f6caeed5c None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:36.006888 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0cc43e-2db9-4454-94c4-036f6caeed5c None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:36.007259 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0cc43e-2db9-4454-94c4-036f6caeed5c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:36.017171 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0cc43e-2db9-4454-94c4-036f6caeed5c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:36.018775 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0cc43e-2db9-4454-94c4-036f6caeed5c None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:36.033956 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-0c0cc43e-2db9-4454-94c4-036f6caeed5c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:06:36.033956 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:36.033956 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:36.033956 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:36.033956 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:36.033956 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:36.033956 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:36.033956 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:36.033956 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:36.033956 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:36.033956 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:36.033956 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:36.033956 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:06:36.033956 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:06:36.033956 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:06:36.035120 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:06:36.035120 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:36.035120 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:36.035120 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:36.035120 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:36.035120 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:36.035120 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:36.035120 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:36.035120 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:36.035120 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:36.035120 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:36.035120 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:36.035120 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:36.035120 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:36.035120 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:36.035120 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:36.035120 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:36.036103 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:06:36.036103 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:06:36.036103 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:06:36.036103 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:06:36.036103 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:06:36.036103 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:06:36.036103 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:06:36.036103 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:36.036834 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 443/871] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jun 21 18:06:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:36.043316 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4327853-6f9d-41c8-b4e7-5364d7d26c50 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:36.063811 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:36.096235 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9ad1d9-0252-440e-b764-551c21053e7d None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:36.096567 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9ad1d9-0252-440e-b764-551c21053e7d None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:36.096878 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9ad1d9-0252-440e-b764-551c21053e7d None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:36.098378 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 444/872] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:06:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:06:36.104273 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd679a7e-2b21-4ca0-ac1f-a88115e1bcbb None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:36.104586 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd679a7e-2b21-4ca0-ac1f-a88115e1bcbb None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:36.104866 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd679a7e-2b21-4ca0-ac1f-a88115e1bcbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:36.125228 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-bd679a7e-2b21-4ca0-ac1f-a88115e1bcbb None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:06:36.126592 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-bd679a7e-2b21-4ca0-ac1f-a88115e1bcbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:06:36.166492 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4327853-6f9d-41c8-b4e7-5364d7d26c50 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:36.182652 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:36.208880 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 445/873] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:06:36 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:06:36.215318 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e254e90b-631a-4b95-9130-fe9f2fe395e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:36.215996 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e254e90b-631a-4b95-9130-fe9f2fe395e3 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:36.216279 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e254e90b-631a-4b95-9130-fe9f2fe395e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:36.216580 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e254e90b-631a-4b95-9130-fe9f2fe395e3 None admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:36.217379 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e254e90b-631a-4b95-9130-fe9f2fe395e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:36.217742 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e254e90b-631a-4b95-9130-fe9f2fe395e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:36.219285 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e254e90b-631a-4b95-9130-fe9f2fe395e3 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:36.230108 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd679a7e-2b21-4ca0-ac1f-a88115e1bcbb None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:36.233094 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 429/874] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jun 21 18:06:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:36.238800 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e254e90b-631a-4b95-9130-fe9f2fe395e3 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:36.311274 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:36.343942 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e254e90b-631a-4b95-9130-fe9f2fe395e3 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:36.345253 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd679a7e-2b21-4ca0-ac1f-a88115e1bcbb None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:36.379729 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fff290-d4a5-4c76-8286-0a007e1245ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:36.380414 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fff290-d4a5-4c76-8286-0a007e1245ae None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:36.380723 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fff290-d4a5-4c76-8286-0a007e1245ae None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:36.381006 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fff290-d4a5-4c76-8286-0a007e1245ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:36.381990 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fff290-d4a5-4c76-8286-0a007e1245ae None admin] RBAC: Adding query filter params (role.id=acf4e26e0ceb41ddad002dd4edf6373c, scope.project.id=2a4699d1ec70456da5f8e0aa4b1439b7, user.id=fcfcd9f134e74b09bfdac337c81c8a07) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:36.382171 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fff290-d4a5-4c76-8286-0a007e1245ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:36.383559 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fff290-d4a5-4c76-8286-0a007e1245ae None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:36.387790 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5373c2-89c7-454a-844d-839558bdd8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:36.388958 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5373c2-89c7-454a-844d-839558bdd8da None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:36.389479 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5373c2-89c7-454a-844d-839558bdd8da None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:36.389952 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5373c2-89c7-454a-844d-839558bdd8da None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:36.390813 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5373c2-89c7-454a-844d-839558bdd8da None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:36.392378 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5373c2-89c7-454a-844d-839558bdd8da None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:36.396681 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 430/875] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jun 21 18:06:36 2022] GET /identity/v3/role_assignments?user.id=fcfcd9f134e74b09bfdac337c81c8a07&scope.project.id=2a4699d1ec70456da5f8e0aa4b1439b7&role.id=acf4e26e0ceb41ddad002dd4edf6373c => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:06:36.410700 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-3b5373c2-89c7-454a-844d-839558bdd8da None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 18:06:36.410700 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:36.410700 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:06:36.410700 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:06:36.410700 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:06:36.410700 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:06:36.410700 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:06:36.410700 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:06:36.410700 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 18:06:36.410700 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:36.410700 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:06:36.410700 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:36.410700 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:36.410700 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:36.410700 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:36.410700 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:36.410700 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:36.411824 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:36.411824 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:36.411824 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:36.411824 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:36.411824 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:36.411824 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:36.411824 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:06:36.411824 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:06:36.411824 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:06:36.411824 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:06:36.411824 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:36.411824 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:36.411824 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:36.411824 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:36.411824 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:36.411824 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:36.412724 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:36.412724 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:36.412724 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:36.412724 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:36.412724 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:36.412724 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:36.412724 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:36.412724 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:36.412724 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:36.412724 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:36.412724 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:06:36.412724 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:06:36.412724 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 18:06:36.412724 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:36.415965 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 446/876] 10.222.0.23 () {56 vars in 1126 bytes} [Tue Jun 21 18:06:36 2022] GET /identity/v3/services/object-store => generated 93 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:36.422544 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fff290-d4a5-4c76-8286-0a007e1245ae None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:36.545444 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fff290-d4a5-4c76-8286-0a007e1245ae None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:36.592074 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6072f2ab-7ad1-4806-93c5-644ffe1eda01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:36.593189 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6072f2ab-7ad1-4806-93c5-644ffe1eda01 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:36.593765 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6072f2ab-7ad1-4806-93c5-644ffe1eda01 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:36.594314 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6072f2ab-7ad1-4806-93c5-644ffe1eda01 None admin] PATH_INFO: `/v3/services` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:36.595230 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6072f2ab-7ad1-4806-93c5-644ffe1eda01 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:36.595740 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6072f2ab-7ad1-4806-93c5-644ffe1eda01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:36.597335 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6072f2ab-7ad1-4806-93c5-644ffe1eda01 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:36.599973 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:36.605005 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 431/877] 10.222.0.23 () {56 vars in 1122 bytes} [Tue Jun 21 18:06:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:36.611716 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5373c2-89c7-454a-844d-839558bdd8da None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:36.664849 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:36.721889 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5373c2-89c7-454a-844d-839558bdd8da None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:36.762399 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e8b745-3840-4157-a498-93cf0b0aeee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:36.763085 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8b745-3840-4157-a498-93cf0b0aeee5 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:36.763369 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8b745-3840-4157-a498-93cf0b0aeee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:36.763650 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8b745-3840-4157-a498-93cf0b0aeee5 None admin] PATH_INFO: `/v3/services` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:36.764377 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e8b745-3840-4157-a498-93cf0b0aeee5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:36.764717 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e8b745-3840-4157-a498-93cf0b0aeee5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:36.766097 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e8b745-3840-4157-a498-93cf0b0aeee5 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:36.771709 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 447/878] 10.222.0.23 () {56 vars in 1122 bytes} [Tue Jun 21 18:06:36 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:06:36.777456 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cabb03-f346-4ff2-8944-45646711cf12 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:36.777748 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cabb03-f346-4ff2-8944-45646711cf12 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:36.778029 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cabb03-f346-4ff2-8944-45646711cf12 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:36.779824 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 432/879] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:06:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:06:36.785714 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa1ab99-3e49-490f-8e49-2911e4a7950f None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:36.786335 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa1ab99-3e49-490f-8e49-2911e4a7950f None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:36.786789 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa1ab99-3e49-490f-8e49-2911e4a7950f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:36.806143 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-7fa1ab99-3e49-490f-8e49-2911e4a7950f None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:06:36.807697 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-7fa1ab99-3e49-490f-8e49-2911e4a7950f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:06:36.886484 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 448/880] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:06:36 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:06:36.909691 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cabb03-f346-4ff2-8944-45646711cf12 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:37.029868 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cabb03-f346-4ff2-8944-45646711cf12 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:37.069124 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3081b04e-eede-4564-890f-3265f0b52c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:37.069814 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3081b04e-eede-4564-890f-3265f0b52c8d None admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:37.070165 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3081b04e-eede-4564-890f-3265f0b52c8d None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:37.070448 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3081b04e-eede-4564-890f-3265f0b52c8d None admin] PATH_INFO: `/v3/services` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:37.071771 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3081b04e-eede-4564-890f-3265f0b52c8d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:37.073248 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3081b04e-eede-4564-890f-3265f0b52c8d None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:37.172496 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 433/881] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Jun 21 18:06:36 2022] POST /identity/v3/services => generated 269 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:06:37.540472 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542dc0d-fb8e-46c0-ae19-ad90d294dfd0 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:37.540906 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542dc0d-fb8e-46c0-ae19-ad90d294dfd0 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:37.541305 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542dc0d-fb8e-46c0-ae19-ad90d294dfd0 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:37.543279 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 449/882] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:06:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:06:37.549621 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288a047-5e35-448c-8cd9-f94bcccaaac1 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:37.550198 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288a047-5e35-448c-8cd9-f94bcccaaac1 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:37.550422 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288a047-5e35-448c-8cd9-f94bcccaaac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:37.569562 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-9288a047-5e35-448c-8cd9-f94bcccaaac1 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:06:37.570797 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-9288a047-5e35-448c-8cd9-f94bcccaaac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:06:37.640778 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:37.663742 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 434/883] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:06:37 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:06:37.685548 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1542dc0d-fb8e-46c0-ae19-ad90d294dfd0 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:37.736906 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:37.803283 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1542dc0d-fb8e-46c0-ae19-ad90d294dfd0 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:37.839752 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99876070-bcc5-4510-93ab-8aad9cfb7bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:37.840613 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99876070-bcc5-4510-93ab-8aad9cfb7bee None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:37.840899 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99876070-bcc5-4510-93ab-8aad9cfb7bee None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:37.841242 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99876070-bcc5-4510-93ab-8aad9cfb7bee None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:37.841939 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99876070-bcc5-4510-93ab-8aad9cfb7bee None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:37.843310 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99876070-bcc5-4510-93ab-8aad9cfb7bee None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:37.856854 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-99876070-bcc5-4510-93ab-8aad9cfb7bee None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 21 18:06:37.856854 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:37.856854 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:06:37.856854 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:06:37.856854 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:06:37.856854 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:06:37.856854 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:06:37.856854 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:06:37.856854 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 21 18:06:37.856854 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:37.856854 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:06:37.856854 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:37.856854 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:37.856854 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:37.856854 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:37.856854 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:37.856854 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:37.857928 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:37.857928 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:37.857928 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:37.857928 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:37.857928 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:37.857928 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:37.857928 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:06:37.857928 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:06:37.857928 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:06:37.857928 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:06:37.857928 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:37.857928 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:37.857928 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:37.857928 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:37.857928 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:37.857928 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:37.858803 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:37.858803 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:37.858803 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:37.858803 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:37.858803 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:37.858803 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:37.858803 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:37.858803 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:37.858803 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:37.858803 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:37.858803 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:06:37.858803 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:06:37.858803 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 21 18:06:37.858803 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:37.864444 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 450/884] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jun 21 18:06:37 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:37.865952 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9288a047-5e35-448c-8cd9-f94bcccaaac1 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:37.980430 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9288a047-5e35-448c-8cd9-f94bcccaaac1 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:38.017300 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42428338-259d-4464-a530-b20a6ccc9ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:38.018258 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42428338-259d-4464-a530-b20a6ccc9ae1 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:38.018703 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42428338-259d-4464-a530-b20a6ccc9ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:38.019129 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42428338-259d-4464-a530-b20a6ccc9ae1 None admin] PATH_INFO: `/v3/services` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:38.020074 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42428338-259d-4464-a530-b20a6ccc9ae1 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:38.020562 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42428338-259d-4464-a530-b20a6ccc9ae1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:38.022112 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42428338-259d-4464-a530-b20a6ccc9ae1 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:38.033977 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 435/885] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jun 21 18:06:37 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:06:38.035703 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99876070-bcc5-4510-93ab-8aad9cfb7bee None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:38.145357 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99876070-bcc5-4510-93ab-8aad9cfb7bee None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:38.148733 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:38.182144 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943d96d3-d2a0-4b30-9b6a-8b6d1a80d3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:38.182821 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d96d3-d2a0-4b30-9b6a-8b6d1a80d3f8 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:38.183101 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d96d3-d2a0-4b30-9b6a-8b6d1a80d3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:38.183379 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d96d3-d2a0-4b30-9b6a-8b6d1a80d3f8 None admin] PATH_INFO: `/v3/services` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:38.184108 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943d96d3-d2a0-4b30-9b6a-8b6d1a80d3f8 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:38.184443 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943d96d3-d2a0-4b30-9b6a-8b6d1a80d3f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:38.185820 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943d96d3-d2a0-4b30-9b6a-8b6d1a80d3f8 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:38.196233 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 451/886] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jun 21 18:06:38 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:38.198299 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42428338-259d-4464-a530-b20a6ccc9ae1 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:38.260945 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:38.310635 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42428338-259d-4464-a530-b20a6ccc9ae1 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:38.346733 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af598e65-486c-4566-8060-0e7c05c0c20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:38.347409 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af598e65-486c-4566-8060-0e7c05c0c20f None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:38.347696 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af598e65-486c-4566-8060-0e7c05c0c20f None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:38.347976 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af598e65-486c-4566-8060-0e7c05c0c20f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:38.349065 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af598e65-486c-4566-8060-0e7c05c0c20f None admin] RBAC: Adding query filter params (interface=public, service_id=058d9e7fe09444e4bc37cc0b7bb79177, region_id=RegionOne) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:38.349436 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af598e65-486c-4566-8060-0e7c05c0c20f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:38.350814 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af598e65-486c-4566-8060-0e7c05c0c20f None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:38.363640 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 436/887] 10.222.0.23 () {56 vars in 1251 bytes} [Tue Jun 21 18:06:38 2022] GET /identity/v3/endpoints?service_id=058d9e7fe09444e4bc37cc0b7bb79177&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:38.365153 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943d96d3-d2a0-4b30-9b6a-8b6d1a80d3f8 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:38.480809 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943d96d3-d2a0-4b30-9b6a-8b6d1a80d3f8 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:38.518675 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc3c175-9a96-4751-bf1c-cfd63dab01cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:38.519353 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc3c175-9a96-4751-bf1c-cfd63dab01cf None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:38.519623 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc3c175-9a96-4751-bf1c-cfd63dab01cf None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:38.519909 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc3c175-9a96-4751-bf1c-cfd63dab01cf None admin] PATH_INFO: `/v3/services` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:38.520640 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc3c175-9a96-4751-bf1c-cfd63dab01cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:38.522011 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc3c175-9a96-4751-bf1c-cfd63dab01cf None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:38.534804 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 452/888] 10.222.0.23 () {56 vars in 1087 bytes} [Tue Jun 21 18:06:38 2022] GET /identity/v3/services => generated 1268 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:06:38.724826 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ced9d4-7070-43b3-ad5e-caa61a6aed9b None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:38.725350 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ced9d4-7070-43b3-ad5e-caa61a6aed9b None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:38.725744 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ced9d4-7070-43b3-ad5e-caa61a6aed9b None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:38.729579 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 437/889] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:06:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:06:38.736793 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c7ba66-ee71-444e-9d10-5638158bf8a7 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:38.737728 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c7ba66-ee71-444e-9d10-5638158bf8a7 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:38.738232 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c7ba66-ee71-444e-9d10-5638158bf8a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:38.765195 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-15c7ba66-ee71-444e-9d10-5638158bf8a7 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:06:38.766820 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-15c7ba66-ee71-444e-9d10-5638158bf8a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:06:38.808436 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:38.878058 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 453/890] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:06:38 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:06:38.930075 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ced9d4-7070-43b3-ad5e-caa61a6aed9b None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:39.065132 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c8695a-1435-4d71-b24b-623c1ceb9efd None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:39.066331 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c8695a-1435-4d71-b24b-623c1ceb9efd None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:39.067155 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c8695a-1435-4d71-b24b-623c1ceb9efd None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:39.072859 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 454/891] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:06:39 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:06:39.082452 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038816a2-a712-454a-8409-c3e3479cf778 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:39.083349 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038816a2-a712-454a-8409-c3e3479cf778 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:39.084102 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038816a2-a712-454a-8409-c3e3479cf778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:39.110553 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ced9d4-7070-43b3-ad5e-caa61a6aed9b None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:39.115176 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-038816a2-a712-454a-8409-c3e3479cf778 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:06:39.117734 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-038816a2-a712-454a-8409-c3e3479cf778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:06:39.129026 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:39.153088 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957f582c-e9ce-4fdd-8a06-f24f668a2a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:39.154178 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f582c-e9ce-4fdd-8a06-f24f668a2a6e None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:39.154619 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f582c-e9ce-4fdd-8a06-f24f668a2a6e None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:39.155096 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f582c-e9ce-4fdd-8a06-f24f668a2a6e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:39.168504 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957f582c-e9ce-4fdd-8a06-f24f668a2a6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:06:39.177367 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-957f582c-e9ce-4fdd-8a06-f24f668a2a6e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 18:06:39.177367 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:39.177367 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:39.177367 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:39.177367 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:39.177367 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:39.177367 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:39.177367 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:39.177367 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:39.177367 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:39.177367 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:39.177367 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:39.177367 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:06:39.177367 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:06:39.177367 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:06:39.177367 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:06:39.178579 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:06:39.178579 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:06:39.178579 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:06:39.178579 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:06:39.178579 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:39.178579 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:39.178579 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:39.178579 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:39.178579 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:39.178579 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:39.178579 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:39.178579 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:39.178579 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:39.178579 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:39.178579 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:39.178579 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:39.179600 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:39.179600 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:39.179600 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:39.179600 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:39.179600 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:06:39.179600 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:06:39.179600 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:06:39.179600 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:06:39.179600 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:06:39.179600 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:06:39.179600 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 18:06:39.179600 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:39.182767 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 438/892] 10.222.0.23 () {56 vars in 1120 bytes} [Tue Jun 21 18:06:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:39.191815 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957f582c-e9ce-4fdd-8a06-f24f668a2a6e None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:39.276560 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 455/893] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:06:39 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:06:39.322437 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038816a2-a712-454a-8409-c3e3479cf778 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:39.395919 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957f582c-e9ce-4fdd-8a06-f24f668a2a6e None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:39.467716 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab76749c-bab2-4902-bd98-0c6df7fd0681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:39.469008 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab76749c-bab2-4902-bd98-0c6df7fd0681 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:39.469533 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab76749c-bab2-4902-bd98-0c6df7fd0681 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:39.470019 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab76749c-bab2-4902-bd98-0c6df7fd0681 None admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:39.471400 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab76749c-bab2-4902-bd98-0c6df7fd0681 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:39.471931 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab76749c-bab2-4902-bd98-0c6df7fd0681 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:39.474758 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab76749c-bab2-4902-bd98-0c6df7fd0681 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:39.496695 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 439/894] 10.222.0.23 () {56 vars in 1115 bytes} [Tue Jun 21 18:06:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:39.499193 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdb5862-8449-4d6f-8b55-8ef78a7588f1 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:39.499613 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdb5862-8449-4d6f-8b55-8ef78a7588f1 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:39.499790 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:39.500095 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdb5862-8449-4d6f-8b55-8ef78a7588f1 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:39.502642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 440/895] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:06:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:06:39.503976 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdb5862-8449-4d6f-8b55-8ef78a7588f1 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:39.522389 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038816a2-a712-454a-8409-c3e3479cf778 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:39.554015 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:39.584945 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479cd461-e2a4-4359-ad29-ace1d0479587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:39.585960 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479cd461-e2a4-4359-ad29-ace1d0479587 None admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:39.586462 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479cd461-e2a4-4359-ad29-ace1d0479587 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:39.587466 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479cd461-e2a4-4359-ad29-ace1d0479587 None admin] PATH_INFO: `/v3/services` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:39.588905 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479cd461-e2a4-4359-ad29-ace1d0479587 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:39.591553 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479cd461-e2a4-4359-ad29-ace1d0479587 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:39.636705 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdb5862-8449-4d6f-8b55-8ef78a7588f1 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:39.678669 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed2acbd-dd5c-41b2-b379-e92b09f3fecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:39.679559 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed2acbd-dd5c-41b2-b379-e92b09f3fecd None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:39.679853 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed2acbd-dd5c-41b2-b379-e92b09f3fecd None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:39.680151 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed2acbd-dd5c-41b2-b379-e92b09f3fecd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:39.691930 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 456/896] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Jun 21 18:06:39 2022] POST /identity/v3/services => generated 241 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:06:39.695007 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d251839-27d8-4957-9110-962af7fd0390 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:39.695889 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d251839-27d8-4957-9110-962af7fd0390 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:39.697234 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d251839-27d8-4957-9110-962af7fd0390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:39.703817 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed2acbd-dd5c-41b2-b379-e92b09f3fecd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:39.707164 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed2acbd-dd5c-41b2-b379-e92b09f3fecd None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:39.731900 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-9d251839-27d8-4957-9110-962af7fd0390 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:06:39.734538 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-9d251839-27d8-4957-9110-962af7fd0390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:06:39.753813 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-bed2acbd-dd5c-41b2-b379-e92b09f3fecd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:39.753813 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:39.753813 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:39.753813 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:39.753813 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:39.753813 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:39.753813 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:39.753813 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:39.753813 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:39.753813 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:39.753813 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:39.753813 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:39.753813 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:06:39.753813 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:06:39.753813 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:06:39.755960 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:06:39.755960 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:39.755960 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:39.755960 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:39.755960 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:39.755960 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:39.755960 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:39.755960 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:39.755960 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:39.755960 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:39.755960 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:39.755960 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:39.755960 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:39.755960 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:39.755960 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:39.755960 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:39.755960 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:39.758398 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:06:39.758398 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:06:39.758398 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:39.758398 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:39.762289 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 441/897] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:06:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:39.768275 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed2acbd-dd5c-41b2-b379-e92b09f3fecd None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:39.916630 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 457/898] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:06:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:06:39.966789 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d251839-27d8-4957-9110-962af7fd0390 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:39.976722 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:39.998245 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed2acbd-dd5c-41b2-b379-e92b09f3fecd None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:40.063532 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1181fbed-dbcb-4d7f-ac8f-8887155b4077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:40.064684 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1181fbed-dbcb-4d7f-ac8f-8887155b4077 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:40.065436 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1181fbed-dbcb-4d7f-ac8f-8887155b4077 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:40.065699 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1181fbed-dbcb-4d7f-ac8f-8887155b4077 None admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:40.067057 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1181fbed-dbcb-4d7f-ac8f-8887155b4077 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:40.067620 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1181fbed-dbcb-4d7f-ac8f-8887155b4077 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:40.069656 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:40.070472 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1181fbed-dbcb-4d7f-ac8f-8887155b4077 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:40.095631 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d251839-27d8-4957-9110-962af7fd0390 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:40.100549 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 442/899] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:06:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:40.103673 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1181fbed-dbcb-4d7f-ac8f-8887155b4077 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:40.136367 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf12e8-9e48-4fb5-a359-2e33cbe94c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:40.137611 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf12e8-9e48-4fb5-a359-2e33cbe94c7a None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:40.138167 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf12e8-9e48-4fb5-a359-2e33cbe94c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:40.138607 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf12e8-9e48-4fb5-a359-2e33cbe94c7a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:40.139580 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bf12e8-9e48-4fb5-a359-2e33cbe94c7a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:40.141195 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bf12e8-9e48-4fb5-a359-2e33cbe94c7a None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:40.155663 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-05bf12e8-9e48-4fb5-a359-2e33cbe94c7a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 21 18:06:40.155663 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:40.155663 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:06:40.155663 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:06:40.155663 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:06:40.155663 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:06:40.155663 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:06:40.155663 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:06:40.155663 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 21 18:06:40.155663 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:40.155663 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:06:40.155663 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:40.155663 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:40.155663 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:40.155663 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:40.155663 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:40.156953 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:40.156953 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:40.156953 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:40.156953 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:40.156953 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:40.156953 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:40.156953 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:40.156953 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:06:40.156953 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:06:40.156953 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:06:40.156953 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:06:40.156953 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:40.156953 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:40.156953 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:40.156953 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:40.156953 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:40.158049 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:40.158049 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:40.158049 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:40.158049 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:40.158049 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:40.158049 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:40.158049 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:40.158049 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:40.158049 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:40.158049 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:40.158049 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:40.158049 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:06:40.158049 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:06:40.158049 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 21 18:06:40.158049 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:40.160744 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 458/900] 10.222.0.23 () {56 vars in 1132 bytes} [Tue Jun 21 18:06:39 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:06:40.170447 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf12e8-9e48-4fb5-a359-2e33cbe94c7a None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:40.286664 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1181fbed-dbcb-4d7f-ac8f-8887155b4077 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:40.353158 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4386c0ff-2bb0-4a5d-976c-7a461d688ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:40.354625 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4386c0ff-2bb0-4a5d-976c-7a461d688ef3 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:40.355131 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4386c0ff-2bb0-4a5d-976c-7a461d688ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:40.355630 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4386c0ff-2bb0-4a5d-976c-7a461d688ef3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:40.370925 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4386c0ff-2bb0-4a5d-976c-7a461d688ef3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:40.373555 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4386c0ff-2bb0-4a5d-976c-7a461d688ef3 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:40.380281 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf12e8-9e48-4fb5-a359-2e33cbe94c7a None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:40.404586 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-4386c0ff-2bb0-4a5d-976c-7a461d688ef3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 18:06:40.404586 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:40.404586 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:40.404586 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:40.404586 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:40.404586 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:40.404586 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:40.404586 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:40.404586 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:40.404586 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:40.404586 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:40.404586 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:40.404586 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:06:40.404586 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:06:40.404586 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:06:40.404586 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:06:40.405614 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:40.405614 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:40.405614 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:06:40.405614 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:06:40.405614 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:06:40.405614 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:06:40.405614 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:40.405614 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:40.405614 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:40.405614 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:40.405614 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:40.405614 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:40.405614 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:40.405614 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:40.405614 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:40.405614 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:40.405614 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:40.405614 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:40.406580 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:40.406580 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:40.406580 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:06:40.406580 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:06:40.406580 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:06:40.406580 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:06:40.406580 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:06:40.406580 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:06:40.406580 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 18:06:40.406580 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:40.412821 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 443/901] 10.222.0.23 () {56 vars in 1117 bytes} [Tue Jun 21 18:06:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:06:40.416696 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4386c0ff-2bb0-4a5d-976c-7a461d688ef3 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:40.440477 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:40.453631 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a5a34-e7ef-4d34-8895-7db2bd54e072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:40.455591 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a5a34-e7ef-4d34-8895-7db2bd54e072 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:40.459757 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a5a34-e7ef-4d34-8895-7db2bd54e072 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:40.459757 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a5a34-e7ef-4d34-8895-7db2bd54e072 None admin] PATH_INFO: `/v3/services` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:40.459757 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65a5a34-e7ef-4d34-8895-7db2bd54e072 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:40.459757 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65a5a34-e7ef-4d34-8895-7db2bd54e072 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:40.462705 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65a5a34-e7ef-4d34-8895-7db2bd54e072 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:40.476510 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 459/902] 10.222.0.23 () {56 vars in 1126 bytes} [Tue Jun 21 18:06:40 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:40.486110 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a5a34-e7ef-4d34-8895-7db2bd54e072 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:40.511708 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:40.629693 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4386c0ff-2bb0-4a5d-976c-7a461d688ef3 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:40.699219 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf30b17-e851-4f85-a8c4-6f61cddc1897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:40.700429 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf30b17-e851-4f85-a8c4-6f61cddc1897 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:40.700864 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf30b17-e851-4f85-a8c4-6f61cddc1897 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:40.701353 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf30b17-e851-4f85-a8c4-6f61cddc1897 None admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:40.702936 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf30b17-e851-4f85-a8c4-6f61cddc1897 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:40.703451 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf30b17-e851-4f85-a8c4-6f61cddc1897 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:40.706296 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf30b17-e851-4f85-a8c4-6f61cddc1897 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:40.724122 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a5a34-e7ef-4d34-8895-7db2bd54e072 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:40.749242 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 444/903] 10.222.0.23 () {56 vars in 1149 bytes} [Tue Jun 21 18:06:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:06:40.758621 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf30b17-e851-4f85-a8c4-6f61cddc1897 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:40.795227 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193ce428-1eb7-4f3b-8a53-5f7631162f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:40.796362 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ce428-1eb7-4f3b-8a53-5f7631162f76 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:40.796892 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ce428-1eb7-4f3b-8a53-5f7631162f76 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:40.797391 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ce428-1eb7-4f3b-8a53-5f7631162f76 None admin] PATH_INFO: `/v3/services` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:40.798735 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193ce428-1eb7-4f3b-8a53-5f7631162f76 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:40.799293 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193ce428-1eb7-4f3b-8a53-5f7631162f76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:40.802028 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193ce428-1eb7-4f3b-8a53-5f7631162f76 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:40.819386 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 460/904] 10.222.0.23 () {56 vars in 1126 bytes} [Tue Jun 21 18:06:40 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:06:40.822742 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193ce428-1eb7-4f3b-8a53-5f7631162f76 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:40.903402 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:40.959564 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:40.964893 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf30b17-e851-4f85-a8c4-6f61cddc1897 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:41.011017 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193ce428-1eb7-4f3b-8a53-5f7631162f76 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:41.026308 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6027d4-5882-495e-8788-2f3f8e7d2299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:41.027446 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6027d4-5882-495e-8788-2f3f8e7d2299 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:41.027952 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6027d4-5882-495e-8788-2f3f8e7d2299 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:41.028480 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6027d4-5882-495e-8788-2f3f8e7d2299 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:41.049566 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e739ace9-ac6e-4f18-8f11-6620bde855ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:41.050248 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e739ace9-ac6e-4f18-8f11-6620bde855ed None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:41.050535 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e739ace9-ac6e-4f18-8f11-6620bde855ed None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:41.050805 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e739ace9-ac6e-4f18-8f11-6620bde855ed None admin] PATH_INFO: `/v3/endpoints` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:41.051604 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e739ace9-ac6e-4f18-8f11-6620bde855ed None admin] RBAC: Adding query filter params (interface=public, service_id=767beae6df0b4a57a684e2ec066ee37f, region_id=RegionOne) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:41.051916 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e739ace9-ac6e-4f18-8f11-6620bde855ed None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:41.053333 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e739ace9-ac6e-4f18-8f11-6620bde855ed None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:41.073355 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 461/905] 10.222.0.23 () {56 vars in 1251 bytes} [Tue Jun 21 18:06:40 2022] GET /identity/v3/endpoints?service_id=767beae6df0b4a57a684e2ec066ee37f&interface=public®ion_id=RegionOne => generated 194 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:41.073355 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25fd88a-f4ef-49ed-bd97-aeba2b5d1fd0 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:41.073355 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25fd88a-f4ef-49ed-bd97-aeba2b5d1fd0 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:41.073355 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25fd88a-f4ef-49ed-bd97-aeba2b5d1fd0 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:41.073355 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 462/906] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:06:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:06:41.076111 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe11ea03-cd2b-4448-8907-d100e0a533aa None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:41.076429 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe11ea03-cd2b-4448-8907-d100e0a533aa None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:41.076705 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe11ea03-cd2b-4448-8907-d100e0a533aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:41.096447 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-fe11ea03-cd2b-4448-8907-d100e0a533aa None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:06:41.097719 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-fe11ea03-cd2b-4448-8907-d100e0a533aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:06:41.159811 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6027d4-5882-495e-8788-2f3f8e7d2299 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:41.162718 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6027d4-5882-495e-8788-2f3f8e7d2299 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:41.177844 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 463/907] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:06:41 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:06:41.186133 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe11ea03-cd2b-4448-8907-d100e0a533aa None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:41.273978 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:41.296917 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:41.317480 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe11ea03-cd2b-4448-8907-d100e0a533aa None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:41.359712 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6fdcad-fef4-41d3-8011-122c731ef25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:41.360412 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6fdcad-fef4-41d3-8011-122c731ef25c None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:41.360734 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6fdcad-fef4-41d3-8011-122c731ef25c None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:41.361025 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6fdcad-fef4-41d3-8011-122c731ef25c None admin] PATH_INFO: `/v3/services` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:41.361738 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6fdcad-fef4-41d3-8011-122c731ef25c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:41.363125 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6fdcad-fef4-41d3-8011-122c731ef25c None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:41.372982 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 445/908] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:06:40 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 607 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 21 18:06:41.372982 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6027d4-5882-495e-8788-2f3f8e7d2299 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:41.379221 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 464/909] 10.222.0.23 () {56 vars in 1087 bytes} [Tue Jun 21 18:06:41 2022] GET /identity/v3/services => generated 1497 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:06:41.381196 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6fdcad-fef4-41d3-8011-122c731ef25c None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:41.544407 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6fdcad-fef4-41d3-8011-122c731ef25c None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:41.573316 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6027d4-5882-495e-8788-2f3f8e7d2299 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:41.606961 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:41.629557 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e861304-4e13-4b74-92bc-e39e4121a852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:41.631852 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e861304-4e13-4b74-92bc-e39e4121a852 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:41.632696 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e861304-4e13-4b74-92bc-e39e4121a852 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:41.633315 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e861304-4e13-4b74-92bc-e39e4121a852 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:41.634525 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b23999-134f-4fcc-922a-2cba6e16c801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:41.635443 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b23999-134f-4fcc-922a-2cba6e16c801 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:41.635772 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b23999-134f-4fcc-922a-2cba6e16c801 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:41.636038 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b23999-134f-4fcc-922a-2cba6e16c801 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:41.636750 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b23999-134f-4fcc-922a-2cba6e16c801 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:41.638172 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b23999-134f-4fcc-922a-2cba6e16c801 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:41.658350 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e861304-4e13-4b74-92bc-e39e4121a852 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:41.661140 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e861304-4e13-4b74-92bc-e39e4121a852 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:41.664228 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-70b23999-134f-4fcc-922a-2cba6e16c801 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 21 18:06:41.664228 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:41.664228 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:06:41.664228 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:06:41.664228 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:06:41.664228 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:06:41.664228 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:06:41.664228 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:06:41.664228 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 21 18:06:41.664228 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:41.664228 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:06:41.664228 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:41.664228 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:41.664228 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:41.664228 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:41.664228 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:41.664228 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:41.667215 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:41.667215 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:41.667215 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:41.667215 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:41.667215 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:41.667215 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:41.667215 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:06:41.667215 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:06:41.667215 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:06:41.667215 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:06:41.667215 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:41.667215 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:41.667215 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:41.667215 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:41.667215 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:41.667215 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:41.670318 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:41.670318 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:41.670318 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:41.670318 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:41.670318 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:41.670318 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:41.670318 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:41.670318 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:41.670318 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:41.670318 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:41.670318 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:06:41.670318 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:06:41.670318 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 21 18:06:41.670318 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:41.670318 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 446/910] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jun 21 18:06:41 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:41.675087 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b23999-134f-4fcc-922a-2cba6e16c801 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:41.691237 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-3e861304-4e13-4b74-92bc-e39e4121a852 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:41.691237 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:41.691237 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:41.691237 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:41.691237 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:41.691237 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:41.691237 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:41.691237 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:41.691237 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:41.691237 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:41.691237 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:41.691237 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:41.691237 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:06:41.691237 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:06:41.691237 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:06:41.693588 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:06:41.693588 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:41.693588 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:41.693588 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:41.693588 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:41.693588 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:41.693588 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:41.693588 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:41.693588 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:41.693588 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:41.693588 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:41.693588 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:41.693588 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:41.693588 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:41.693588 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:41.693588 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:41.693588 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:41.695351 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:06:41.695351 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:06:41.695351 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:41.695351 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:41.695351 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 465/911] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:06:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:41.705806 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e861304-4e13-4b74-92bc-e39e4121a852 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:41.737022 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:41.827259 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e861304-4e13-4b74-92bc-e39e4121a852 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:41.874720 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0779437-63a3-42dc-84e4-794e609b69f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:41.875695 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0779437-63a3-42dc-84e4-794e609b69f4 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:41.876141 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0779437-63a3-42dc-84e4-794e609b69f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:41.876601 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0779437-63a3-42dc-84e4-794e609b69f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:41.877499 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0779437-63a3-42dc-84e4-794e609b69f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:41.877963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0779437-63a3-42dc-84e4-794e609b69f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:41.880026 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0779437-63a3-42dc-84e4-794e609b69f4 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:41.885041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b23999-134f-4fcc-922a-2cba6e16c801 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:41.900551 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 466/912] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:06:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:41.914158 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0779437-63a3-42dc-84e4-794e609b69f4 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:41.941407 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd69746-9cd6-48e3-987a-6669ee19e683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:41.942119 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd69746-9cd6-48e3-987a-6669ee19e683 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:41.942861 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd69746-9cd6-48e3-987a-6669ee19e683 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:41.943149 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd69746-9cd6-48e3-987a-6669ee19e683 None admin] PATH_INFO: `/v3/services` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:41.943876 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd69746-9cd6-48e3-987a-6669ee19e683 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:41.944353 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd69746-9cd6-48e3-987a-6669ee19e683 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:41.945895 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd69746-9cd6-48e3-987a-6669ee19e683 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:41.952750 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 447/913] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jun 21 18:06:41 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:06:41.959837 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd69746-9cd6-48e3-987a-6669ee19e683 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:42.000632 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:42.088244 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0779437-63a3-42dc-84e4-794e609b69f4 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:42.112620 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:42.124329 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd69746-9cd6-48e3-987a-6669ee19e683 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:42.145487 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8c9d6b-7901-43d6-b6d8-7e3f291fa656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:42.146772 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c9d6b-7901-43d6-b6d8-7e3f291fa656 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:42.147395 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c9d6b-7901-43d6-b6d8-7e3f291fa656 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:42.147853 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c9d6b-7901-43d6-b6d8-7e3f291fa656 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:42.162537 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8c9d6b-7901-43d6-b6d8-7e3f291fa656 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:42.166498 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8c9d6b-7901-43d6-b6d8-7e3f291fa656 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:42.179936 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74f4650-066d-4c7e-87a2-e98057249d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:42.181323 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f4650-066d-4c7e-87a2-e98057249d77 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:42.181714 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f4650-066d-4c7e-87a2-e98057249d77 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:42.182179 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f4650-066d-4c7e-87a2-e98057249d77 None admin] PATH_INFO: `/v3/services` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:42.183062 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74f4650-066d-4c7e-87a2-e98057249d77 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:42.183369 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74f4650-066d-4c7e-87a2-e98057249d77 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:42.184794 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74f4650-066d-4c7e-87a2-e98057249d77 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:42.187658 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-8f8c9d6b-7901-43d6-b6d8-7e3f291fa656 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:06:42.187658 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:42.187658 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:42.187658 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:42.187658 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:42.187658 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:42.187658 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:42.187658 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:42.187658 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:42.187658 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:42.187658 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:42.187658 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:42.187658 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:06:42.187658 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:06:42.187658 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:06:42.189579 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:06:42.189579 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:42.189579 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:42.189579 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:42.189579 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:42.189579 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:42.189579 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:42.189579 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:42.189579 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:42.189579 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:42.189579 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:42.189579 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:42.189579 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:42.189579 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:42.189579 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:42.189579 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:42.189579 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:42.191360 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:06:42.191360 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:06:42.191360 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:06:42.191360 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:06:42.191360 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:06:42.191360 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:06:42.191360 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:06:42.191360 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:42.191360 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 467/914] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jun 21 18:06:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:42.192326 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18313491-f4c2-4344-8693-eda3e8662b4a None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:42.192326 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18313491-f4c2-4344-8693-eda3e8662b4a None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:42.193391 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18313491-f4c2-4344-8693-eda3e8662b4a None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:42.195022 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 468/915] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:06:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:06:42.196028 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18313491-f4c2-4344-8693-eda3e8662b4a None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:42.199329 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 448/916] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jun 21 18:06:41 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:06:42.204614 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f86bd7-e264-448f-aa81-28096b8529af None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:42.205488 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f86bd7-e264-448f-aa81-28096b8529af None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:42.206199 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f86bd7-e264-448f-aa81-28096b8529af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:42.239264 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-66f86bd7-e264-448f-aa81-28096b8529af None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:06:42.242109 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-66f86bd7-e264-448f-aa81-28096b8529af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:06:42.328623 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 449/917] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:06:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:06:42.328623 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f86bd7-e264-448f-aa81-28096b8529af None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:42.349192 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:42.356119 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18313491-f4c2-4344-8693-eda3e8662b4a None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:42.410211 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36455f5c-17b1-4ac6-8854-c0e6e49e82f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:42.411322 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36455f5c-17b1-4ac6-8854-c0e6e49e82f3 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:42.411780 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36455f5c-17b1-4ac6-8854-c0e6e49e82f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:42.412220 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36455f5c-17b1-4ac6-8854-c0e6e49e82f3 None admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:42.413607 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36455f5c-17b1-4ac6-8854-c0e6e49e82f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:42.414109 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36455f5c-17b1-4ac6-8854-c0e6e49e82f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:42.418274 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36455f5c-17b1-4ac6-8854-c0e6e49e82f3 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:42.445322 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 469/918] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jun 21 18:06:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:42.445322 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36455f5c-17b1-4ac6-8854-c0e6e49e82f3 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:42.469832 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:42.483282 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f86bd7-e264-448f-aa81-28096b8529af None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:42.523859 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f80ec-a03f-4093-b2a0-2a639f95ced5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:42.524979 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f80ec-a03f-4093-b2a0-2a639f95ced5 None admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:42.525445 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f80ec-a03f-4093-b2a0-2a639f95ced5 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:42.525693 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f80ec-a03f-4093-b2a0-2a639f95ced5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:42.526636 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1f80ec-a03f-4093-b2a0-2a639f95ced5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:42.527857 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1f80ec-a03f-4093-b2a0-2a639f95ced5 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:42.574508 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36455f5c-17b1-4ac6-8854-c0e6e49e82f3 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:42.615149 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae55f0e-d4ae-450e-85a6-ecaddc70900a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:42.615903 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae55f0e-d4ae-450e-85a6-ecaddc70900a None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:42.616182 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae55f0e-d4ae-450e-85a6-ecaddc70900a None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:42.616486 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae55f0e-d4ae-450e-85a6-ecaddc70900a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:42.628835 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 450/919] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Jun 21 18:06:42 2022] POST /identity/v3/endpoints => generated 346 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:06:42.629064 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae55f0e-d4ae-450e-85a6-ecaddc70900a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:42.629150 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f80ec-a03f-4093-b2a0-2a639f95ced5 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:42.629256 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae55f0e-d4ae-450e-85a6-ecaddc70900a None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:42.645684 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-bae55f0e-d4ae-450e-85a6-ecaddc70900a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 18:06:42.645684 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:42.645684 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:06:42.645684 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:06:42.645684 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:06:42.645684 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:06:42.645684 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:06:42.645684 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:06:42.645684 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 18:06:42.645684 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:42.645684 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:06:42.645684 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:42.645684 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:42.645684 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:42.645684 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:42.645684 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:42.645684 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:42.646856 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:42.646856 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:42.646856 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:42.646856 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:42.646856 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:42.646856 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:42.646856 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:06:42.646856 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:06:42.646856 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:06:42.646856 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:06:42.646856 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:42.646856 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:42.646856 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:42.646856 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:42.646856 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:42.646856 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:42.647710 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:42.647710 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:42.647710 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:42.647710 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:42.647710 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:42.647710 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:42.647710 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:42.647710 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:42.647710 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:42.647710 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:42.647710 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:06:42.647710 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:06:42.647710 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 18:06:42.647710 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:42.647710 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 470/920] 10.222.0.23 () {56 vars in 1126 bytes} [Tue Jun 21 18:06:42 2022] GET /identity/v3/services/object-store => generated 93 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:42.661782 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae55f0e-d4ae-450e-85a6-ecaddc70900a None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:42.710559 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:42.784553 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f80ec-a03f-4093-b2a0-2a639f95ced5 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:42.834588 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0004c057-a4c7-4a64-a120-0f25b93b541d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:42.836158 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0004c057-a4c7-4a64-a120-0f25b93b541d None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:42.837934 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0004c057-a4c7-4a64-a120-0f25b93b541d None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:42.837934 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0004c057-a4c7-4a64-a120-0f25b93b541d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:42.838940 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0004c057-a4c7-4a64-a120-0f25b93b541d None admin] RBAC: Adding query filter params (role.id=e950e7aaae294af383fc9e83eed8250f, scope.project.id=2a4699d1ec70456da5f8e0aa4b1439b7, user.id=fcfcd9f134e74b09bfdac337c81c8a07) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:42.839491 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0004c057-a4c7-4a64-a120-0f25b93b541d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:42.843775 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0004c057-a4c7-4a64-a120-0f25b93b541d None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:42.850991 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae55f0e-d4ae-450e-85a6-ecaddc70900a None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:42.861248 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 451/921] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jun 21 18:06:42 2022] GET /identity/v3/role_assignments?user.id=fcfcd9f134e74b09bfdac337c81c8a07&scope.project.id=2a4699d1ec70456da5f8e0aa4b1439b7&role.id=e950e7aaae294af383fc9e83eed8250f => generated 259 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:42.910054 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbfb3f7-1672-4a92-9281-91534466716a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:42.910740 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbfb3f7-1672-4a92-9281-91534466716a None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:42.911021 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbfb3f7-1672-4a92-9281-91534466716a None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:42.911298 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbfb3f7-1672-4a92-9281-91534466716a None admin] PATH_INFO: `/v3/services` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:42.912039 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbfb3f7-1672-4a92-9281-91534466716a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:42.912344 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbfb3f7-1672-4a92-9281-91534466716a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:42.913821 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbfb3f7-1672-4a92-9281-91534466716a None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:42.920491 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 471/922] 10.222.0.23 () {56 vars in 1122 bytes} [Tue Jun 21 18:06:42 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:42.929755 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0004c057-a4c7-4a64-a120-0f25b93b541d None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:42.952811 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:43.111507 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0004c057-a4c7-4a64-a120-0f25b93b541d None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:43.164577 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e72d1-a2d3-4912-97f7-50696e855d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:43.165824 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e72d1-a2d3-4912-97f7-50696e855d53 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:43.166393 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e72d1-a2d3-4912-97f7-50696e855d53 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:43.166393 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e72d1-a2d3-4912-97f7-50696e855d53 None admin] PATH_INFO: `/v3/services` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:43.167037 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1e72d1-a2d3-4912-97f7-50696e855d53 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:43.167347 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1e72d1-a2d3-4912-97f7-50696e855d53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:43.168771 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1e72d1-a2d3-4912-97f7-50696e855d53 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:43.174490 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 452/923] 10.222.0.23 () {56 vars in 1122 bytes} [Tue Jun 21 18:06:42 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:43.179995 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbfb3f7-1672-4a92-9281-91534466716a None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:43.287793 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:43.300940 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbfb3f7-1672-4a92-9281-91534466716a None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:43.336138 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9167df99-1c1f-4c8f-aab3-cfa10fb36adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:43.337361 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9167df99-1c1f-4c8f-aab3-cfa10fb36adb None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:43.338252 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9167df99-1c1f-4c8f-aab3-cfa10fb36adb None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:43.338536 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9167df99-1c1f-4c8f-aab3-cfa10fb36adb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:43.339533 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9167df99-1c1f-4c8f-aab3-cfa10fb36adb None admin] RBAC: Adding query filter params (interface=public, service_id=5e03d9146bcf463e9fcade22442438da, region_id=RegionOne) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:43.339921 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9167df99-1c1f-4c8f-aab3-cfa10fb36adb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:43.341843 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9167df99-1c1f-4c8f-aab3-cfa10fb36adb None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:43.349414 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 472/924] 10.222.0.23 () {56 vars in 1251 bytes} [Tue Jun 21 18:06:43 2022] GET /identity/v3/endpoints?service_id=5e03d9146bcf463e9fcade22442438da&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:43.355661 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e72d1-a2d3-4912-97f7-50696e855d53 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:43.429544 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:43.480425 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e72d1-a2d3-4912-97f7-50696e855d53 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:43.517648 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323abb34-ab52-4a07-b756-52902814584b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:43.518709 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323abb34-ab52-4a07-b756-52902814584b None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:43.519205 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323abb34-ab52-4a07-b756-52902814584b None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:43.519685 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323abb34-ab52-4a07-b756-52902814584b None admin] PATH_INFO: `/v3/services` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:43.520620 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323abb34-ab52-4a07-b756-52902814584b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:43.522233 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323abb34-ab52-4a07-b756-52902814584b None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:43.530287 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 453/925] 10.222.0.23 () {56 vars in 1087 bytes} [Tue Jun 21 18:06:43 2022] GET /identity/v3/services => generated 1497 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:06:44.144956 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdba6a5-23d2-4362-becf-1001f40d8eb2 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:44.145159 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdba6a5-23d2-4362-becf-1001f40d8eb2 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:44.145439 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdba6a5-23d2-4362-becf-1001f40d8eb2 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:44.147162 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 473/926] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:06:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:06:44.154028 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da83bf-80da-4ce8-86bb-a05759f07069 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:44.154579 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da83bf-80da-4ce8-86bb-a05759f07069 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:44.154969 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da83bf-80da-4ce8-86bb-a05759f07069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:44.184129 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-38da83bf-80da-4ce8-86bb-a05759f07069 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:06:44.186299 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-38da83bf-80da-4ce8-86bb-a05759f07069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:06:44.324886 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 454/927] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:06:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:06:44.347382 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdba6a5-23d2-4362-becf-1001f40d8eb2 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:44.467215 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdba6a5-23d2-4362-becf-1001f40d8eb2 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:44.504982 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4934a411-7f76-4a0c-abaa-624992f46507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:44.505809 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4934a411-7f76-4a0c-abaa-624992f46507 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:44.506090 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4934a411-7f76-4a0c-abaa-624992f46507 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:44.506364 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4934a411-7f76-4a0c-abaa-624992f46507 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:44.507030 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4934a411-7f76-4a0c-abaa-624992f46507 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:44.508434 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4934a411-7f76-4a0c-abaa-624992f46507 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:44.522507 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-4934a411-7f76-4a0c-abaa-624992f46507 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 21 18:06:44.522507 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:44.522507 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:06:44.522507 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:06:44.522507 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:06:44.522507 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:06:44.522507 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:06:44.522507 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:06:44.522507 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 21 18:06:44.522507 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:44.522507 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:06:44.522507 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:44.522507 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:44.522507 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:44.522507 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:44.522507 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:44.524520 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:44.524520 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:44.524520 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:44.524520 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:44.524520 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:44.524520 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:44.524520 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:44.524520 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:06:44.524520 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:06:44.524520 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:06:44.524520 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:06:44.524520 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:44.524520 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:44.524520 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:44.524520 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:44.524520 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:44.526252 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:44.526252 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:44.526252 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:44.526252 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:44.526252 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:44.526252 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:44.526252 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:44.526252 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:44.526252 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:44.526252 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:44.526252 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:44.526252 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:06:44.526252 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:06:44.526252 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 21 18:06:44.526252 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:44.526252 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 474/928] 10.222.0.23 () {56 vars in 1132 bytes} [Tue Jun 21 18:06:44 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:44.532426 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38da83bf-80da-4ce8-86bb-a05759f07069 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:44.660815 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:44.711829 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38da83bf-80da-4ce8-86bb-a05759f07069 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:44.751036 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939fe6c9-b44a-4d6f-99ac-a25f23e5ae15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:44.751989 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939fe6c9-b44a-4d6f-99ac-a25f23e5ae15 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:44.752456 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939fe6c9-b44a-4d6f-99ac-a25f23e5ae15 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:44.752891 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939fe6c9-b44a-4d6f-99ac-a25f23e5ae15 None admin] PATH_INFO: `/v3/services` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:44.753796 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939fe6c9-b44a-4d6f-99ac-a25f23e5ae15 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:44.754256 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939fe6c9-b44a-4d6f-99ac-a25f23e5ae15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:44.755841 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939fe6c9-b44a-4d6f-99ac-a25f23e5ae15 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:44.762868 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 455/929] 10.222.0.23 () {56 vars in 1126 bytes} [Tue Jun 21 18:06:44 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:06:44.772257 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4934a411-7f76-4a0c-abaa-624992f46507 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:44.794699 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:44.890348 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2c3f3-f02c-43f6-a2f8-cb8d8f80e13c None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:44.891333 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2c3f3-f02c-43f6-a2f8-cb8d8f80e13c None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:44.893141 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2c3f3-f02c-43f6-a2f8-cb8d8f80e13c None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:44.896819 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 456/930] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:06:44 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:06:44.902118 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee7d626-aafb-44c9-bd77-e9df0a21ec88 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:44.902509 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee7d626-aafb-44c9-bd77-e9df0a21ec88 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:44.902666 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee7d626-aafb-44c9-bd77-e9df0a21ec88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:44.905051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4934a411-7f76-4a0c-abaa-624992f46507 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:44.921621 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-6ee7d626-aafb-44c9-bd77-e9df0a21ec88 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:06:44.922859 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-6ee7d626-aafb-44c9-bd77-e9df0a21ec88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:06:44.981437 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db4cc3d-31f6-4f15-806f-ef0ada9f3805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:44.983128 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4cc3d-31f6-4f15-806f-ef0ada9f3805 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:44.983912 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4cc3d-31f6-4f15-806f-ef0ada9f3805 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:44.984709 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4cc3d-31f6-4f15-806f-ef0ada9f3805 None admin] PATH_INFO: `/v3/services` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:44.986350 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db4cc3d-31f6-4f15-806f-ef0ada9f3805 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:44.987189 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db4cc3d-31f6-4f15-806f-ef0ada9f3805 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:44.990097 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db4cc3d-31f6-4f15-806f-ef0ada9f3805 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:45.000916 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 475/931] 10.222.0.23 () {56 vars in 1126 bytes} [Tue Jun 21 18:06:44 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:45.007207 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db4cc3d-31f6-4f15-806f-ef0ada9f3805 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:45.010590 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 457/932] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:06:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:06:45.034962 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee7d626-aafb-44c9-bd77-e9df0a21ec88 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:45.144489 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:45.146418 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:45.182487 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db4cc3d-31f6-4f15-806f-ef0ada9f3805 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:45.200862 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee7d626-aafb-44c9-bd77-e9df0a21ec88 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:45.242827 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b46b273-8c6e-464a-a875-0965a6cf18ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:45.244527 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46b273-8c6e-464a-a875-0965a6cf18ae None admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:45.245145 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46b273-8c6e-464a-a875-0965a6cf18ae None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:45.245741 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46b273-8c6e-464a-a875-0965a6cf18ae None admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:45.246679 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b46b273-8c6e-464a-a875-0965a6cf18ae None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:45.248426 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b46b273-8c6e-464a-a875-0965a6cf18ae None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:45.250267 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: WARNING keystone.server.flask.common [None req-4b46b273-8c6e-464a-a875-0965a6cf18ae None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 21 18:06:45.251321 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0339a725-115d-4740-97bd-dccd93d25ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:45.252539 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0339a725-115d-4740-97bd-dccd93d25ebb None admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:45.253047 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0339a725-115d-4740-97bd-dccd93d25ebb None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:45.253544 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0339a725-115d-4740-97bd-dccd93d25ebb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:45.255006 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0339a725-115d-4740-97bd-dccd93d25ebb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:45.257944 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0339a725-115d-4740-97bd-dccd93d25ebb None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:45.305065 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.sql.core [None req-4b46b273-8c6e-464a-a875-0965a6cf18ae None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 21 18:06:45.305065 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 21 18:06:45.305065 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [parameters: {'id': '16040544b1ab449591182c95833ce2de', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 21 18:06:45.305065 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80615) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 21 18:06:45.313754 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-4b46b273-8c6e-464a-a875-0965a6cf18ae None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 16040544b1ab449591182c95833ce2de.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 16040544b1ab449591182c95833ce2de. Jun 21 18:06:45.313754 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:45.313754 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 21 18:06:45.313754 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 21 18:06:45.313754 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 21 18:06:45.313754 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 21 18:06:45.313754 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 21 18:06:45.313754 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application result = self._query(query) Jun 21 18:06:45.313754 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 21 18:06:45.313754 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application conn.query(q) Jun 21 18:06:45.313754 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 21 18:06:45.313754 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 21 18:06:45.313754 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 21 18:06:45.316719 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application result.read() Jun 21 18:06:45.316719 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 21 18:06:45.316719 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 21 18:06:45.316719 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 21 18:06:45.316719 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 21 18:06:45.316719 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 21 18:06:45.316719 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 21 18:06:45.316719 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 21 18:06:45.316719 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 21 18:06:45.316719 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 21 18:06:45.316719 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:45.316719 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 21 18:06:45.316719 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:45.316719 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:45.316719 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 21 18:06:45.316719 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 21 18:06:45.316719 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 21 18:06:45.316719 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 21 18:06:45.318615 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 21 18:06:45.318615 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application next(self.gen) Jun 21 18:06:45.318615 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 21 18:06:45.318615 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application yield resource Jun 21 18:06:45.318615 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 21 18:06:45.318615 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application next(self.gen) Jun 21 18:06:45.318615 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 21 18:06:45.318615 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application self.session.rollback() Jun 21 18:06:45.318615 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 21 18:06:45.318615 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application self.force_reraise() Jun 21 18:06:45.318615 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 21 18:06:45.318615 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise self.value Jun 21 18:06:45.318615 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 21 18:06:45.318615 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 21 18:06:45.318615 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 21 18:06:45.318615 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application session.commit() Jun 21 18:06:45.318615 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 21 18:06:45.321753 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 21 18:06:45.321753 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 21 18:06:45.321753 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application self._prepare_impl() Jun 21 18:06:45.321753 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 21 18:06:45.321753 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application self.session.flush() Jun 21 18:06:45.321753 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 21 18:06:45.321753 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application self._flush(objects) Jun 21 18:06:45.321753 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 21 18:06:45.321753 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 21 18:06:45.321753 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 21 18:06:45.321753 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application compat.raise_( Jun 21 18:06:45.321753 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 21 18:06:45.321753 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception Jun 21 18:06:45.321753 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 21 18:06:45.321753 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application flush_context.execute() Jun 21 18:06:45.321753 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 21 18:06:45.321753 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rec.execute(self) Jun 21 18:06:45.324130 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 21 18:06:45.324130 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 21 18:06:45.324130 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 21 18:06:45.324130 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 21 18:06:45.324130 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 21 18:06:45.324130 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 21 18:06:45.324130 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 21 18:06:45.324130 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 21 18:06:45.324130 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 21 18:06:45.324130 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 21 18:06:45.324130 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 21 18:06:45.324130 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 21 18:06:45.324130 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 21 18:06:45.324130 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 21 18:06:45.324130 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 21 18:06:45.326088 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 21 18:06:45.326088 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 21 18:06:45.326088 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception Jun 21 18:06:45.326088 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 21 18:06:45.326088 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 21 18:06:45.326088 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 21 18:06:45.326088 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 21 18:06:45.326088 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 21 18:06:45.326088 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application result = self._query(query) Jun 21 18:06:45.326088 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 21 18:06:45.326088 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application conn.query(q) Jun 21 18:06:45.326088 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 21 18:06:45.326088 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 21 18:06:45.326088 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 21 18:06:45.326088 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application result.read() Jun 21 18:06:45.326088 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 21 18:06:45.326088 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 21 18:06:45.330323 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 21 18:06:45.330323 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 21 18:06:45.330323 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 21 18:06:45.330323 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 21 18:06:45.330323 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 21 18:06:45.330323 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 21 18:06:45.330323 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 21 18:06:45.330323 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 21 18:06:45.330323 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [parameters: {'id': '16040544b1ab449591182c95833ce2de', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 21 18:06:45.330323 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 21 18:06:45.330323 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:45.330323 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:06:45.330323 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:45.330323 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:45.330323 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 21 18:06:45.332223 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 21 18:06:45.332223 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 21 18:06:45.332223 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 21 18:06:45.332223 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 21 18:06:45.332223 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:45.332223 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:06:45.332223 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:45.332223 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:45.332223 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:45.332223 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:45.332223 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:45.332223 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:45.332223 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:45.332223 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:45.332223 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:45.332223 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:45.332223 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:45.334208 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:45.334208 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 21 18:06:45.334208 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 21 18:06:45.334208 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:45.334208 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:45.334208 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 21 18:06:45.334208 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 21 18:06:45.334208 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 16040544b1ab449591182c95833ce2de. Jun 21 18:06:45.334208 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:45.334208 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 458/933] 10.222.0.23 () {60 vars in 1140 bytes} [Tue Jun 21 18:06:45 2022] POST /identity/v3/projects => generated 262 bytes in 281 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:45.334208 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57abfa-770e-40ea-8a76-399a9e6f92a7 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:45.334208 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57abfa-770e-40ea-8a76-399a9e6f92a7 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:45.335593 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57abfa-770e-40ea-8a76-399a9e6f92a7 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:45.335593 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 459/934] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:06:45 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:06:45.335593 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a57abfa-770e-40ea-8a76-399a9e6f92a7 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:45.372676 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 476/935] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Jun 21 18:06:45 2022] POST /identity/v3/endpoints => generated 347 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:06:45.374047 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d82f17-025d-4450-83e5-b4325a9d6d54 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:45.374411 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d82f17-025d-4450-83e5-b4325a9d6d54 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:45.374694 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d82f17-025d-4450-83e5-b4325a9d6d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:45.398160 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-75d82f17-025d-4450-83e5-b4325a9d6d54 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:06:45.399476 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-75d82f17-025d-4450-83e5-b4325a9d6d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:06:45.508049 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 477/936] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:06:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:06:45.509896 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a57abfa-770e-40ea-8a76-399a9e6f92a7 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:45.533085 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d82f17-025d-4450-83e5-b4325a9d6d54 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:45.535953 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:45.578316 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694ef7d2-8b5e-48b2-9748-b6dc28ac11a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:45.579816 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694ef7d2-8b5e-48b2-9748-b6dc28ac11a6 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:45.580451 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694ef7d2-8b5e-48b2-9748-b6dc28ac11a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:45.581026 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694ef7d2-8b5e-48b2-9748-b6dc28ac11a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:45.595402 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694ef7d2-8b5e-48b2-9748-b6dc28ac11a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:45.598736 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694ef7d2-8b5e-48b2-9748-b6dc28ac11a6 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:45.612408 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:45.628807 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-694ef7d2-8b5e-48b2-9748-b6dc28ac11a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:06:45.628807 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:45.628807 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:45.628807 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:45.628807 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:45.628807 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:45.628807 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:45.628807 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:45.628807 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:45.628807 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:45.628807 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:45.628807 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:45.628807 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:06:45.628807 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:06:45.628807 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:06:45.630943 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:06:45.630943 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:45.630943 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:45.630943 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:45.630943 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:45.630943 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:45.630943 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:45.630943 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:45.630943 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:45.630943 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:45.630943 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:45.630943 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:45.630943 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:45.630943 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:45.630943 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:45.630943 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:45.630943 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:45.633084 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:06:45.633084 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:06:45.633084 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:06:45.633084 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:06:45.633084 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:06:45.633084 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:06:45.633084 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:06:45.633084 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:45.642979 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 460/937] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jun 21 18:06:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:45.644102 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694ef7d2-8b5e-48b2-9748-b6dc28ac11a6 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:45.670161 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d82f17-025d-4450-83e5-b4325a9d6d54 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:45.744425 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b931b9b2-82d3-4115-b981-9388b8531e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:45.744425 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b931b9b2-82d3-4115-b981-9388b8531e77 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:45.744425 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b931b9b2-82d3-4115-b981-9388b8531e77 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:45.744425 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b931b9b2-82d3-4115-b981-9388b8531e77 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:45.756853 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b931b9b2-82d3-4115-b981-9388b8531e77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:06:45.769959 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-b931b9b2-82d3-4115-b981-9388b8531e77 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 18:06:45.769959 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:45.769959 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:45.769959 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:45.769959 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:45.769959 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:45.769959 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:45.769959 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:45.769959 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:45.769959 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:45.769959 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:45.769959 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:45.769959 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:06:45.769959 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:06:45.769959 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:06:45.769959 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:06:45.772015 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:06:45.772015 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:06:45.772015 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:06:45.772015 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:06:45.772015 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:45.772015 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:45.772015 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:45.772015 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:45.772015 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:45.772015 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:45.772015 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:45.772015 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:45.772015 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:45.772015 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:45.772015 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:45.772015 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:45.773869 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:45.773869 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:45.773869 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:45.773869 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:45.773869 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:06:45.773869 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:06:45.773869 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:06:45.773869 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:06:45.773869 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:06:45.773869 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:06:45.773869 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 18:06:45.773869 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:45.776482 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 478/938] 10.222.0.23 () {56 vars in 1120 bytes} [Tue Jun 21 18:06:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:45.786633 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b931b9b2-82d3-4115-b981-9388b8531e77 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:45.815812 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694ef7d2-8b5e-48b2-9748-b6dc28ac11a6 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:45.858917 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7e9ea1-fc57-49ba-a3d2-721562714f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:45.859602 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7e9ea1-fc57-49ba-a3d2-721562714f79 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:45.859887 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7e9ea1-fc57-49ba-a3d2-721562714f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:45.860165 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7e9ea1-fc57-49ba-a3d2-721562714f79 None admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:45.860970 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7e9ea1-fc57-49ba-a3d2-721562714f79 None admin] RBAC: Adding query filter params (name=service) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:45.861277 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7e9ea1-fc57-49ba-a3d2-721562714f79 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:45.862729 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7e9ea1-fc57-49ba-a3d2-721562714f79 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:45.886416 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 461/939] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Jun 21 18:06:45 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:06:45.996330 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b931b9b2-82d3-4115-b981-9388b8531e77 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:46.042695 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:46.065289 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a7ad7e-2133-404b-a19b-d016865ca2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:46.066670 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a7ad7e-2133-404b-a19b-d016865ca2da None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:46.067324 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a7ad7e-2133-404b-a19b-d016865ca2da None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:46.068010 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a7ad7e-2133-404b-a19b-d016865ca2da None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:46.069746 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a7ad7e-2133-404b-a19b-d016865ca2da None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:46.070489 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a7ad7e-2133-404b-a19b-d016865ca2da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:46.074153 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a7ad7e-2133-404b-a19b-d016865ca2da None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:46.093447 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 479/940] 10.222.0.23 () {56 vars in 1115 bytes} [Tue Jun 21 18:06:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:06:46.102944 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7e9ea1-fc57-49ba-a3d2-721562714f79 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:46.127847 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b6cbf5-74c7-4f32-8f7f-d22391aed737 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:46.128639 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b6cbf5-74c7-4f32-8f7f-d22391aed737 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:46.129403 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b6cbf5-74c7-4f32-8f7f-d22391aed737 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:46.132265 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 480/941] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:06:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:06:46.140130 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0141879-e6f9-4395-8853-567dc98e10c7 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:46.140951 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0141879-e6f9-4395-8853-567dc98e10c7 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:46.141635 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0141879-e6f9-4395-8853-567dc98e10c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:46.160729 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:46.173532 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-d0141879-e6f9-4395-8853-567dc98e10c7 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:06:46.175150 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-d0141879-e6f9-4395-8853-567dc98e10c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:06:46.257552 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7e9ea1-fc57-49ba-a3d2-721562714f79 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:46.295169 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 481/942] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:06:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:06:46.305933 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3ac2b9-5aaa-49c1-8cd9-db222f9ce19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:46.306837 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3ac2b9-5aaa-49c1-8cd9-db222f9ce19b None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:46.307198 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3ac2b9-5aaa-49c1-8cd9-db222f9ce19b None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:46.307550 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3ac2b9-5aaa-49c1-8cd9-db222f9ce19b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:46.320090 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3ac2b9-5aaa-49c1-8cd9-db222f9ce19b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:46.322247 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3ac2b9-5aaa-49c1-8cd9-db222f9ce19b None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:46.335566 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0141879-e6f9-4395-8853-567dc98e10c7 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:46.355735 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-3e3ac2b9-5aaa-49c1-8cd9-db222f9ce19b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:46.355735 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:46.355735 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:46.355735 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:46.355735 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:46.355735 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:46.355735 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:46.355735 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:46.355735 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:46.355735 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:46.355735 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:46.355735 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:46.355735 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:06:46.355735 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:06:46.355735 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:06:46.357749 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:06:46.357749 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:46.357749 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:46.357749 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:46.357749 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:46.357749 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:46.357749 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:46.357749 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:46.357749 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:46.357749 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:46.357749 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:46.357749 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:46.357749 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:46.357749 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:46.357749 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:46.357749 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:46.357749 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:46.359062 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:06:46.359062 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:06:46.359062 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:46.359062 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:46.359062 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 462/943] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:06:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:46.370352 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3ac2b9-5aaa-49c1-8cd9-db222f9ce19b None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:46.520940 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:46.532275 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0141879-e6f9-4395-8853-567dc98e10c7 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:46.551227 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3ac2b9-5aaa-49c1-8cd9-db222f9ce19b None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:46.559385 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:46.595941 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb45afe5-7f37-4d7d-bb60-08f269fd7a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:46.597256 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb45afe5-7f37-4d7d-bb60-08f269fd7a75 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:46.597729 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb45afe5-7f37-4d7d-bb60-08f269fd7a75 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:46.598203 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb45afe5-7f37-4d7d-bb60-08f269fd7a75 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:46.599765 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb45afe5-7f37-4d7d-bb60-08f269fd7a75 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:46.602947 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb45afe5-7f37-4d7d-bb60-08f269fd7a75 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:46.619720 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cef8ff-4d08-4951-9280-42684a4905e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:46.621323 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cef8ff-4d08-4951-9280-42684a4905e2 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:46.622061 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cef8ff-4d08-4951-9280-42684a4905e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:46.622782 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cef8ff-4d08-4951-9280-42684a4905e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:46.624481 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cef8ff-4d08-4951-9280-42684a4905e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:46.625250 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cef8ff-4d08-4951-9280-42684a4905e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:46.626642 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-bb45afe5-7f37-4d7d-bb60-08f269fd7a75 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 18:06:46.626642 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:46.626642 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:06:46.626642 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:06:46.626642 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:06:46.626642 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:06:46.626642 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:06:46.626642 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:06:46.626642 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 18:06:46.626642 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:46.626642 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:06:46.626642 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:46.626642 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:46.626642 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:46.626642 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:46.626642 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:46.626642 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:46.628543 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:46.628543 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:46.628543 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:46.628543 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:46.628543 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:46.628543 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:46.628543 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:06:46.628543 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:06:46.628543 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:06:46.628543 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:06:46.628543 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:46.628543 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:46.628543 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:46.628543 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:46.628543 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:46.628543 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:46.630206 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:46.630206 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:46.630206 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:46.630206 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:46.630206 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:46.630206 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:46.630206 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:46.630206 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:46.630206 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:46.630206 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:46.630206 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:06:46.630206 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:06:46.630206 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 18:06:46.630206 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:46.634494 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cef8ff-4d08-4951-9280-42684a4905e2 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:46.636860 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 482/944] 10.222.0.23 () {56 vars in 1126 bytes} [Tue Jun 21 18:06:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:06:46.638446 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb45afe5-7f37-4d7d-bb60-08f269fd7a75 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:46.664531 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 463/945] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:06:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:06:46.667741 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cef8ff-4d08-4951-9280-42684a4905e2 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:46.826817 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb45afe5-7f37-4d7d-bb60-08f269fd7a75 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:46.853852 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cef8ff-4d08-4951-9280-42684a4905e2 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:46.905637 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8167bb-bad7-49bd-b018-964d80d173ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:46.906709 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8167bb-bad7-49bd-b018-964d80d173ff None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:46.907187 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8167bb-bad7-49bd-b018-964d80d173ff None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:46.907661 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8167bb-bad7-49bd-b018-964d80d173ff None admin] PATH_INFO: `/v3/services` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:46.908982 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8167bb-bad7-49bd-b018-964d80d173ff None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:46.909513 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8167bb-bad7-49bd-b018-964d80d173ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:46.912067 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8167bb-bad7-49bd-b018-964d80d173ff None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:46.920380 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88502891-c4aa-4351-8628-928517f25387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:46.922174 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88502891-c4aa-4351-8628-928517f25387 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:46.922939 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88502891-c4aa-4351-8628-928517f25387 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:46.923756 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88502891-c4aa-4351-8628-928517f25387 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:46.928047 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 483/946] 10.222.0.23 () {56 vars in 1122 bytes} [Tue Jun 21 18:06:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:46.929571 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8167bb-bad7-49bd-b018-964d80d173ff None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:46.941927 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88502891-c4aa-4351-8628-928517f25387 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:46.945060 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88502891-c4aa-4351-8628-928517f25387 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:46.977099 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-88502891-c4aa-4351-8628-928517f25387 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 18:06:46.977099 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:46.977099 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:46.977099 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:46.977099 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:46.977099 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:46.977099 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:46.977099 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:46.977099 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:46.977099 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:46.977099 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:46.977099 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:46.977099 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:06:46.977099 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:06:46.977099 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:06:46.977099 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:06:46.979526 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:46.979526 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:46.979526 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:06:46.979526 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:06:46.979526 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:06:46.979526 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:06:46.979526 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:46.979526 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:46.979526 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:46.979526 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:46.979526 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:46.979526 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:46.979526 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:46.979526 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:46.979526 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:46.979526 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:46.979526 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:46.979526 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:46.981480 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:46.981480 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:46.981480 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:06:46.981480 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:06:46.981480 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:06:46.981480 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:06:46.981480 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:06:46.981480 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:06:46.981480 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 18:06:46.981480 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:46.992615 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 464/947] 10.222.0.23 () {56 vars in 1117 bytes} [Tue Jun 21 18:06:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:06:46.997151 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88502891-c4aa-4351-8628-928517f25387 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:47.003542 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:47.009196 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:47.108599 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8167bb-bad7-49bd-b018-964d80d173ff None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:47.167184 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec23f2c4-fc0b-4eba-b26c-23697b15bcee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:47.168605 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec23f2c4-fc0b-4eba-b26c-23697b15bcee None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:47.169301 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec23f2c4-fc0b-4eba-b26c-23697b15bcee None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:47.169938 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec23f2c4-fc0b-4eba-b26c-23697b15bcee None admin] PATH_INFO: `/v3/services` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:47.171358 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec23f2c4-fc0b-4eba-b26c-23697b15bcee None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:47.172068 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec23f2c4-fc0b-4eba-b26c-23697b15bcee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:47.174705 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec23f2c4-fc0b-4eba-b26c-23697b15bcee None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:47.184345 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 484/948] 10.222.0.23 () {56 vars in 1122 bytes} [Tue Jun 21 18:06:46 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:47.191412 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec23f2c4-fc0b-4eba-b26c-23697b15bcee None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:47.195633 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88502891-c4aa-4351-8628-928517f25387 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:47.260779 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649190db-1860-41df-814d-60db1029a9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:47.261977 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649190db-1860-41df-814d-60db1029a9df None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:47.262340 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649190db-1860-41df-814d-60db1029a9df None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:47.262815 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649190db-1860-41df-814d-60db1029a9df None admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:47.264304 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649190db-1860-41df-814d-60db1029a9df None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:47.264851 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649190db-1860-41df-814d-60db1029a9df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:47.267389 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649190db-1860-41df-814d-60db1029a9df None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:47.310037 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 465/949] 10.222.0.23 () {56 vars in 1149 bytes} [Tue Jun 21 18:06:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:06:47.313511 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649190db-1860-41df-814d-60db1029a9df None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:47.381320 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec23f2c4-fc0b-4eba-b26c-23697b15bcee None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:47.396699 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:47.407363 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:47.451549 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66a8554-23e3-435b-8a7f-731d4fd3a1e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:47.452700 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a8554-23e3-435b-8a7f-731d4fd3a1e3 None admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:47.453197 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a8554-23e3-435b-8a7f-731d4fd3a1e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:47.453688 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a8554-23e3-435b-8a7f-731d4fd3a1e3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:47.455242 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66a8554-23e3-435b-8a7f-731d4fd3a1e3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:47.457902 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66a8554-23e3-435b-8a7f-731d4fd3a1e3 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:47.460007 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649190db-1860-41df-814d-60db1029a9df None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:47.505138 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61273722-0862-48a7-b2e2-ccb09fd483b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:47.505952 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61273722-0862-48a7-b2e2-ccb09fd483b9 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:47.506240 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61273722-0862-48a7-b2e2-ccb09fd483b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:47.506518 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61273722-0862-48a7-b2e2-ccb09fd483b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:47.520038 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61273722-0862-48a7-b2e2-ccb09fd483b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:47.521538 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61273722-0862-48a7-b2e2-ccb09fd483b9 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:47.542242 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-61273722-0862-48a7-b2e2-ccb09fd483b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:47.542242 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:47.542242 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:47.542242 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:47.542242 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:47.542242 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:47.542242 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:47.542242 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:47.542242 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:47.542242 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:47.542242 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:47.542242 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:47.542242 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:06:47.542242 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:06:47.542242 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:06:47.543245 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:06:47.543245 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:47.543245 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:47.543245 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:47.543245 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:47.543245 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:47.543245 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:47.543245 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:47.543245 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:47.543245 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:47.543245 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:47.543245 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:47.543245 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:47.543245 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:47.543245 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:47.543245 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:47.543245 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:47.544154 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:06:47.544154 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:06:47.544154 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:47.544154 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:47.544154 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 466/950] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:06:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:06:47.550492 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61273722-0862-48a7-b2e2-ccb09fd483b9 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:47.558917 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 485/951] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Jun 21 18:06:47 2022] POST /identity/v3/endpoints => generated 349 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:06:47.687015 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61273722-0862-48a7-b2e2-ccb09fd483b9 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:47.705345 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:47.739718 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a81fb-3f02-4656-b909-c379494b4ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:47.740424 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a81fb-3f02-4656-b909-c379494b4ee6 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:47.740716 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a81fb-3f02-4656-b909-c379494b4ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:47.740993 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a81fb-3f02-4656-b909-c379494b4ee6 None admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:47.741738 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380a81fb-3f02-4656-b909-c379494b4ee6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:47.742049 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380a81fb-3f02-4656-b909-c379494b4ee6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:47.743945 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707ae64-41d6-4995-89e8-5e2e76a3b80b None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:47.744103 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380a81fb-3f02-4656-b909-c379494b4ee6 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:47.744289 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707ae64-41d6-4995-89e8-5e2e76a3b80b None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:47.745399 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707ae64-41d6-4995-89e8-5e2e76a3b80b None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:47.748259 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 486/952] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:06:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:06:47.754226 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc910e9-92b9-4c1b-882d-b257b45b4028 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:47.754547 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc910e9-92b9-4c1b-882d-b257b45b4028 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:47.754771 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc910e9-92b9-4c1b-882d-b257b45b4028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:47.759571 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 467/953] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:06:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:06:47.770374 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a81fb-3f02-4656-b909-c379494b4ee6 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:47.782692 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-3cc910e9-92b9-4c1b-882d-b257b45b4028 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:06:47.783933 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-3cc910e9-92b9-4c1b-882d-b257b45b4028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:06:47.880308 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 487/954] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:06:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:06:47.906136 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc910e9-92b9-4c1b-882d-b257b45b4028 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:47.984265 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a81fb-3f02-4656-b909-c379494b4ee6 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:48.018379 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc910e9-92b9-4c1b-882d-b257b45b4028 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:48.019998 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:48.047012 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66759ab4-df7e-4ae4-b665-ab3f9e604f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:48.048362 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66759ab4-df7e-4ae4-b665-ab3f9e604f1c None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:48.048887 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66759ab4-df7e-4ae4-b665-ab3f9e604f1c None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:48.049368 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66759ab4-df7e-4ae4-b665-ab3f9e604f1c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:48.062644 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66759ab4-df7e-4ae4-b665-ab3f9e604f1c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:48.065377 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66759ab4-df7e-4ae4-b665-ab3f9e604f1c None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:48.077029 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd8feb1-4868-474c-bc48-8208014587d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:48.078031 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd8feb1-4868-474c-bc48-8208014587d3 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:48.078426 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd8feb1-4868-474c-bc48-8208014587d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:48.078772 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd8feb1-4868-474c-bc48-8208014587d3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:48.079520 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd8feb1-4868-474c-bc48-8208014587d3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:48.081028 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd8feb1-4868-474c-bc48-8208014587d3 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:48.094505 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-66759ab4-df7e-4ae4-b665-ab3f9e604f1c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:06:48.094505 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:48.094505 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:48.094505 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:48.094505 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:48.094505 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:48.094505 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:48.094505 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:48.094505 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:48.094505 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:48.094505 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:48.094505 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:48.094505 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:06:48.094505 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:06:48.094505 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:06:48.095657 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:06:48.095657 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:48.095657 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:48.095657 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:48.095657 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:48.095657 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:48.095657 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:48.095657 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:48.095657 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:48.095657 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:48.095657 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:48.095657 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:48.095657 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:48.095657 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:48.095657 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:48.095657 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:48.095657 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:48.096671 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:06:48.096671 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:06:48.096671 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:06:48.096671 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:06:48.096671 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:06:48.096671 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:06:48.096671 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:06:48.096671 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:48.097684 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-4dd8feb1-4868-474c-bc48-8208014587d3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 21 18:06:48.097684 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:48.097684 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:06:48.097684 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:06:48.097684 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:06:48.097684 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:06:48.097684 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:06:48.097684 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:06:48.097684 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 21 18:06:48.097684 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:48.097684 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:06:48.097684 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:48.097684 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:48.097684 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:48.097684 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:48.097684 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:48.097684 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:48.099738 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:48.099738 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:48.099738 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:48.099738 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:48.099738 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:48.099738 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:48.099738 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:06:48.099738 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:06:48.099738 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:06:48.099738 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:06:48.099738 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:48.099738 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:48.099738 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:48.099738 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:48.099738 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:48.099738 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:48.100847 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:48.100847 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:48.100847 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:48.100847 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:48.100847 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:48.100847 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:48.100847 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:48.100847 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:48.100847 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:48.100847 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:48.100847 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:06:48.100847 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:06:48.100847 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 21 18:06:48.100847 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:48.100847 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 488/955] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jun 21 18:06:47 2022] GET /identity/v3/services/compute => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:06:48.101947 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 468/956] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jun 21 18:06:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:06:48.112418 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd8feb1-4868-474c-bc48-8208014587d3 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:48.112631 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66759ab4-df7e-4ae4-b665-ab3f9e604f1c None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:48.149130 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:48.244901 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66759ab4-df7e-4ae4-b665-ab3f9e604f1c None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:48.309687 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd8feb1-4868-474c-bc48-8208014587d3 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:48.323789 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a72736-3af0-427a-a895-e30019378f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:48.325364 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a72736-3af0-427a-a895-e30019378f3e None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:48.325803 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a72736-3af0-427a-a895-e30019378f3e None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:48.326249 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a72736-3af0-427a-a895-e30019378f3e None admin] PATH_INFO: `/v3/services` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:48.327668 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a72736-3af0-427a-a895-e30019378f3e None admin] RBAC: Adding query filter params (name=compute) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:48.328144 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a72736-3af0-427a-a895-e30019378f3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:48.331732 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a72736-3af0-427a-a895-e30019378f3e None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:48.347010 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 469/957] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Jun 21 18:06:48 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:48.353448 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a72736-3af0-427a-a895-e30019378f3e None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:48.358366 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a771543-a203-44a9-82c7-00b46605da66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:48.359070 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a771543-a203-44a9-82c7-00b46605da66 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:48.359358 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a771543-a203-44a9-82c7-00b46605da66 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:48.359625 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a771543-a203-44a9-82c7-00b46605da66 None admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:48.360467 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a771543-a203-44a9-82c7-00b46605da66 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:48.360787 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a771543-a203-44a9-82c7-00b46605da66 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:48.362333 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a771543-a203-44a9-82c7-00b46605da66 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:48.385054 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 489/958] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jun 21 18:06:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:48.386517 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a771543-a203-44a9-82c7-00b46605da66 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:48.426998 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:48.511416 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:48.519517 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a72736-3af0-427a-a895-e30019378f3e None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:48.542644 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a771543-a203-44a9-82c7-00b46605da66 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:48.564307 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3368907e-025b-4f32-b2eb-94061a77e009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:48.565032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3368907e-025b-4f32-b2eb-94061a77e009 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:48.565319 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3368907e-025b-4f32-b2eb-94061a77e009 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:48.565595 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3368907e-025b-4f32-b2eb-94061a77e009 None admin] PATH_INFO: `/v3/services` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:48.566413 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3368907e-025b-4f32-b2eb-94061a77e009 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:48.566775 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3368907e-025b-4f32-b2eb-94061a77e009 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:48.568194 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3368907e-025b-4f32-b2eb-94061a77e009 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:48.575012 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 470/959] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Jun 21 18:06:48 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:48.576550 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cfbc0f-1fc0-4111-9a94-c25265cab0b4 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:48.576845 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cfbc0f-1fc0-4111-9a94-c25265cab0b4 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:48.577120 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cfbc0f-1fc0-4111-9a94-c25265cab0b4 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:48.578686 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 471/960] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:06:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:06:48.584504 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4027480-8e93-4e97-9264-f5597d8c5f98 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:48.584791 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4027480-8e93-4e97-9264-f5597d8c5f98 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:48.585168 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4027480-8e93-4e97-9264-f5597d8c5f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:48.602330 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac94a44-1a1b-435b-871b-8c581bb9c463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:48.604457 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac94a44-1a1b-435b-871b-8c581bb9c463 None admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:48.604787 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-e4027480-8e93-4e97-9264-f5597d8c5f98 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:06:48.605188 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac94a44-1a1b-435b-871b-8c581bb9c463 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:48.605669 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac94a44-1a1b-435b-871b-8c581bb9c463 None admin] PATH_INFO: `/v3/projects/2a4699d1ec70456da5f8e0aa4b1439b7/users/fcfcd9f134e74b09bfdac337c81c8a07/roles/e950e7aaae294af383fc9e83eed8250f` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:48.606365 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-e4027480-8e93-4e97-9264-f5597d8c5f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:06:48.609064 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac94a44-1a1b-435b-871b-8c581bb9c463 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a4699d1ec70456da5f8e0aa4b1439b7, user_id=fcfcd9f134e74b09bfdac337c81c8a07, role_id=e950e7aaae294af383fc9e83eed8250f)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:48.611809 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac94a44-1a1b-435b-871b-8c581bb9c463 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:48.693959 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 472/961] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:06:48 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:06:48.713734 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4027480-8e93-4e97-9264-f5597d8c5f98 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:48.721254 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 490/962] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jun 21 18:06:48 2022] PUT /identity/v3/projects/2a4699d1ec70456da5f8e0aa4b1439b7/users/fcfcd9f134e74b09bfdac337c81c8a07/roles/e950e7aaae294af383fc9e83eed8250f => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:06:48.823911 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:48.848277 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4027480-8e93-4e97-9264-f5597d8c5f98 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:48.911947 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce10b3-2dcb-441f-99f6-d69358eae496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:48.913168 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce10b3-2dcb-441f-99f6-d69358eae496 None admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:48.913669 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce10b3-2dcb-441f-99f6-d69358eae496 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:48.914154 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce10b3-2dcb-441f-99f6-d69358eae496 None admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:48.915564 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ce10b3-2dcb-441f-99f6-d69358eae496 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:48.917851 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ce10b3-2dcb-441f-99f6-d69358eae496 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:48.970861 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.sql.core [None req-99ce10b3-2dcb-441f-99f6-d69358eae496 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 21 18:06:48.970861 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 21 18:06:48.970861 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [parameters: {'user_id': '6d762fae44f64a7bbb5927045ab30e70', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 21 18:06:48.970861 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80615) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 21 18:06:48.984581 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-99ce10b3-2dcb-441f-99f6-d69358eae496 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 21 18:06:48.984581 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:48.984581 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 21 18:06:48.984581 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 21 18:06:48.984581 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 21 18:06:48.984581 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 21 18:06:48.984581 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 21 18:06:48.984581 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application result = self._query(query) Jun 21 18:06:48.984581 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 21 18:06:48.984581 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application conn.query(q) Jun 21 18:06:48.984581 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 21 18:06:48.984581 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 21 18:06:48.984581 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 21 18:06:48.984581 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application result.read() Jun 21 18:06:48.984581 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 21 18:06:48.986681 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 21 18:06:48.986681 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 21 18:06:48.986681 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 21 18:06:48.986681 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 21 18:06:48.986681 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 21 18:06:48.986681 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 21 18:06:48.986681 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 21 18:06:48.986681 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 21 18:06:48.986681 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:48.986681 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 21 18:06:48.986681 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:48.986681 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:48.986681 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 21 18:06:48.986681 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 21 18:06:48.986681 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 21 18:06:48.986681 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 21 18:06:48.986681 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 21 18:06:48.986681 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application next(self.gen) Jun 21 18:06:48.987662 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 21 18:06:48.987662 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application yield resource Jun 21 18:06:48.987662 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 21 18:06:48.987662 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application next(self.gen) Jun 21 18:06:48.987662 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 21 18:06:48.987662 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application self.session.rollback() Jun 21 18:06:48.987662 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 21 18:06:48.987662 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application self.force_reraise() Jun 21 18:06:48.987662 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 21 18:06:48.987662 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise self.value Jun 21 18:06:48.987662 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 21 18:06:48.987662 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 21 18:06:48.987662 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 21 18:06:48.987662 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application session.commit() Jun 21 18:06:48.987662 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 21 18:06:48.987662 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 21 18:06:48.987662 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 21 18:06:48.989169 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application self._prepare_impl() Jun 21 18:06:48.989169 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 21 18:06:48.989169 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application self.session.flush() Jun 21 18:06:48.989169 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 21 18:06:48.989169 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application self._flush(objects) Jun 21 18:06:48.989169 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 21 18:06:48.989169 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 21 18:06:48.989169 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 21 18:06:48.989169 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application compat.raise_( Jun 21 18:06:48.989169 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 21 18:06:48.989169 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception Jun 21 18:06:48.989169 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 21 18:06:48.989169 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application flush_context.execute() Jun 21 18:06:48.989169 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 21 18:06:48.989169 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rec.execute(self) Jun 21 18:06:48.989169 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 21 18:06:48.989169 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 21 18:06:48.990129 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 21 18:06:48.990129 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 21 18:06:48.990129 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 21 18:06:48.990129 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 21 18:06:48.990129 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 21 18:06:48.990129 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 21 18:06:48.990129 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 21 18:06:48.990129 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 21 18:06:48.990129 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 21 18:06:48.990129 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 21 18:06:48.990129 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 21 18:06:48.990129 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 21 18:06:48.990129 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 21 18:06:48.990129 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 21 18:06:48.990129 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 21 18:06:48.990129 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception Jun 21 18:06:48.991023 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 21 18:06:48.991023 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 21 18:06:48.991023 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 21 18:06:48.991023 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 21 18:06:48.991023 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 21 18:06:48.991023 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application result = self._query(query) Jun 21 18:06:48.991023 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 21 18:06:48.991023 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application conn.query(q) Jun 21 18:06:48.991023 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 21 18:06:48.991023 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 21 18:06:48.991023 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 21 18:06:48.991023 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application result.read() Jun 21 18:06:48.991023 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 21 18:06:48.991023 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 21 18:06:48.991023 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 21 18:06:48.991023 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 21 18:06:48.991908 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 21 18:06:48.991908 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 21 18:06:48.991908 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 21 18:06:48.991908 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 21 18:06:48.991908 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 21 18:06:48.991908 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 21 18:06:48.991908 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [parameters: {'user_id': '6d762fae44f64a7bbb5927045ab30e70', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 21 18:06:48.991908 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 21 18:06:48.991908 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:48.991908 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:06:48.991908 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:48.991908 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:48.991908 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:48.991908 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:48.991908 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:48.992796 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:48.992796 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:48.992796 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:48.992796 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:48.992796 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:48.992796 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:48.992796 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:48.992796 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 21 18:06:48.992796 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 21 18:06:48.992796 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:48.992796 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:48.992796 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:06:48.992796 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:06:48.992796 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:06:48.992796 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:06:48.992796 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 21 18:06:48.992796 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 21 18:06:48.994007 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 21 18:06:48.994007 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 21 18:06:48.994007 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 21 18:06:48.994007 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 21 18:06:48.994007 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 21 18:06:48.994007 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:48.994007 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 473/963] 10.222.0.23 () {60 vars in 1131 bytes} [Tue Jun 21 18:06:48 2022] POST /identity/v3/users => generated 160 bytes in 274 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:48.998916 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac94a44-1a1b-435b-871b-8c581bb9c463 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:49.139201 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:49.182859 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac94a44-1a1b-435b-871b-8c581bb9c463 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:49.235986 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77c8ee6-5afb-4d50-a2fd-33512e7912c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:49.237291 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c8ee6-5afb-4d50-a2fd-33512e7912c0 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:49.237785 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c8ee6-5afb-4d50-a2fd-33512e7912c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:49.238261 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c8ee6-5afb-4d50-a2fd-33512e7912c0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:49.248185 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77c8ee6-5afb-4d50-a2fd-33512e7912c0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:49.250033 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77c8ee6-5afb-4d50-a2fd-33512e7912c0 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:49.266823 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-d77c8ee6-5afb-4d50-a2fd-33512e7912c0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 21 18:06:49.266823 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:49.266823 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:49.266823 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:49.266823 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:49.266823 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:49.266823 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:49.266823 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:49.266823 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:49.266823 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:49.266823 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:49.266823 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:49.266823 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:06:49.266823 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:06:49.266823 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:06:49.266823 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:06:49.267944 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:49.267944 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:49.267944 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:06:49.267944 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:06:49.267944 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:06:49.267944 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:06:49.267944 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:49.267944 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:49.267944 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:49.267944 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:49.267944 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:49.267944 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:49.267944 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:49.267944 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:49.267944 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:49.267944 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:49.267944 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:49.267944 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:49.269045 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:49.269045 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:49.269045 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:06:49.269045 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:06:49.269045 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:06:49.269045 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:06:49.269045 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:06:49.269045 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:06:49.269045 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 21 18:06:49.269045 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:49.271472 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 491/964] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:06:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:49.281462 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce10b3-2dcb-441f-99f6-d69358eae496 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:49.444111 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce10b3-2dcb-441f-99f6-d69358eae496 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:49.525473 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8422d5af-1e77-43b6-962d-c51b8a01198d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:49.526627 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8422d5af-1e77-43b6-962d-c51b8a01198d None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:49.527124 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8422d5af-1e77-43b6-962d-c51b8a01198d None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:49.527593 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8422d5af-1e77-43b6-962d-c51b8a01198d None admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:49.529200 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8422d5af-1e77-43b6-962d-c51b8a01198d None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:49.529754 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8422d5af-1e77-43b6-962d-c51b8a01198d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:49.532951 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8422d5af-1e77-43b6-962d-c51b8a01198d None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:49.613724 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 474/965] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jun 21 18:06:49 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:50.172299 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ce2d7-7074-44b5-9a97-7f9ce73da957 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:50.172904 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ce2d7-7074-44b5-9a97-7f9ce73da957 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:50.173319 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ce2d7-7074-44b5-9a97-7f9ce73da957 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:50.175771 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 492/966] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:06:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:06:50.182281 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77062310-59a1-48d7-a53e-baff54474646 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:50.182557 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77062310-59a1-48d7-a53e-baff54474646 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:50.182869 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77062310-59a1-48d7-a53e-baff54474646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:50.202543 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-77062310-59a1-48d7-a53e-baff54474646 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:06:50.204090 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-77062310-59a1-48d7-a53e-baff54474646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:06:50.206822 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:50.315461 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 475/967] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:06:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:06:50.358528 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09ce2d7-7074-44b5-9a97-7f9ce73da957 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:50.554080 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09ce2d7-7074-44b5-9a97-7f9ce73da957 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:50.622396 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baa885e-5f05-446c-bfde-8496e208e4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:50.623680 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa885e-5f05-446c-bfde-8496e208e4bc None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:50.624187 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa885e-5f05-446c-bfde-8496e208e4bc None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:50.624685 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa885e-5f05-446c-bfde-8496e208e4bc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:50.625895 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baa885e-5f05-446c-bfde-8496e208e4bc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:50.628558 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baa885e-5f05-446c-bfde-8496e208e4bc None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:50.634637 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:50.654365 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-9baa885e-5f05-446c-bfde-8496e208e4bc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 18:06:50.654365 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:50.654365 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:06:50.654365 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:06:50.654365 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:06:50.654365 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:06:50.654365 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:06:50.654365 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:06:50.654365 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 18:06:50.654365 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:50.654365 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:06:50.654365 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:50.654365 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:50.654365 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:50.654365 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:50.654365 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:50.654365 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:50.656874 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:50.656874 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:50.656874 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:50.656874 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:50.656874 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:50.656874 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:50.656874 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:06:50.656874 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:06:50.656874 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:06:50.656874 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:06:50.656874 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:50.656874 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:50.656874 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:50.656874 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:50.656874 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:50.656874 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:50.658677 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:50.658677 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:50.658677 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:50.658677 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:50.658677 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:50.658677 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:50.658677 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:50.658677 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:50.658677 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:50.658677 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:50.658677 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:06:50.658677 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:06:50.658677 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 18:06:50.658677 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:50.664656 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 493/968] 10.222.0.23 () {56 vars in 1126 bytes} [Tue Jun 21 18:06:50 2022] GET /identity/v3/services/object-store => generated 93 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:50.667621 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77062310-59a1-48d7-a53e-baff54474646 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:50.870345 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77062310-59a1-48d7-a53e-baff54474646 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:50.932467 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddf5ad5-ab2f-42d2-b540-4e7b39174fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:50.933633 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf5ad5-ab2f-42d2-b540-4e7b39174fe7 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:50.934142 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf5ad5-ab2f-42d2-b540-4e7b39174fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:50.934644 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf5ad5-ab2f-42d2-b540-4e7b39174fe7 None admin] PATH_INFO: `/v3/services` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:50.935995 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddf5ad5-ab2f-42d2-b540-4e7b39174fe7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:50.936586 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddf5ad5-ab2f-42d2-b540-4e7b39174fe7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:50.939182 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddf5ad5-ab2f-42d2-b540-4e7b39174fe7 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:50.953923 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 476/969] 10.222.0.23 () {56 vars in 1122 bytes} [Tue Jun 21 18:06:50 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:06:50.955717 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baa885e-5f05-446c-bfde-8496e208e4bc None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:50.966309 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0748e9a5-5644-4a7a-9b2f-3b46688ae833 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:50.966744 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0748e9a5-5644-4a7a-9b2f-3b46688ae833 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:50.966890 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0748e9a5-5644-4a7a-9b2f-3b46688ae833 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:50.968534 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 477/970] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:06:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:06:50.974301 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41319f99-38b3-4bde-ae8e-d71f1d34ad2d None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:50.975493 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41319f99-38b3-4bde-ae8e-d71f1d34ad2d None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:50.975493 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41319f99-38b3-4bde-ae8e-d71f1d34ad2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:50.982548 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:50.996635 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-41319f99-38b3-4bde-ae8e-d71f1d34ad2d None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:06:50.997952 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-41319f99-38b3-4bde-ae8e-d71f1d34ad2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:06:51.100801 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baa885e-5f05-446c-bfde-8496e208e4bc None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:51.128703 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 478/971] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:06:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:06:51.131019 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28198e-d6b5-4017-9970-dd431b342117 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:51.131455 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28198e-d6b5-4017-9970-dd431b342117 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:51.131897 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28198e-d6b5-4017-9970-dd431b342117 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:51.134667 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 479/972] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:06:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:06:51.144103 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb67136b-80d3-4efe-b819-2bce7e2bd038 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:51.144864 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb67136b-80d3-4efe-b819-2bce7e2bd038 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:51.145372 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb67136b-80d3-4efe-b819-2bce7e2bd038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:51.154265 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4de92a-6f49-47fb-8695-07e3acac7e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:51.155254 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4de92a-6f49-47fb-8695-07e3acac7e39 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:51.155678 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4de92a-6f49-47fb-8695-07e3acac7e39 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:51.156082 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4de92a-6f49-47fb-8695-07e3acac7e39 None admin] PATH_INFO: `/v3/services` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:51.157189 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4de92a-6f49-47fb-8695-07e3acac7e39 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:51.157666 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4de92a-6f49-47fb-8695-07e3acac7e39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:51.159349 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4de92a-6f49-47fb-8695-07e3acac7e39 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:51.167519 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 494/973] 10.222.0.23 () {56 vars in 1122 bytes} [Tue Jun 21 18:06:50 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:51.168645 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4de92a-6f49-47fb-8695-07e3acac7e39 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:51.175546 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-fb67136b-80d3-4efe-b819-2bce7e2bd038 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:06:51.176948 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-fb67136b-80d3-4efe-b819-2bce7e2bd038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:06:51.247441 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:51.292448 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 480/974] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:06:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:06:51.292448 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb67136b-80d3-4efe-b819-2bce7e2bd038 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:51.300370 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4de92a-6f49-47fb-8695-07e3acac7e39 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:51.342252 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b7d904-ddb0-47c5-b4cb-1591c00e92c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:51.342945 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b7d904-ddb0-47c5-b4cb-1591c00e92c7 None admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:51.343224 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b7d904-ddb0-47c5-b4cb-1591c00e92c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:51.343501 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b7d904-ddb0-47c5-b4cb-1591c00e92c7 None admin] PATH_INFO: `/v3/services` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:51.344236 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b7d904-ddb0-47c5-b4cb-1591c00e92c7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:51.345663 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b7d904-ddb0-47c5-b4cb-1591c00e92c7 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:51.380413 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:51.421493 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb67136b-80d3-4efe-b819-2bce7e2bd038 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:51.445984 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 495/975] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Jun 21 18:06:51 2022] POST /identity/v3/services => generated 242 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:06:51.445984 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b7d904-ddb0-47c5-b4cb-1591c00e92c7 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:51.470703 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49647e9e-a547-4e2f-84b8-416e44e4aef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:51.471578 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49647e9e-a547-4e2f-84b8-416e44e4aef8 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:51.472081 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49647e9e-a547-4e2f-84b8-416e44e4aef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:51.472544 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49647e9e-a547-4e2f-84b8-416e44e4aef8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:51.473579 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49647e9e-a547-4e2f-84b8-416e44e4aef8 None admin] RBAC: Adding query filter params (interface=admin, service_id=5e03d9146bcf463e9fcade22442438da, region_id=RegionOne) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:51.474295 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49647e9e-a547-4e2f-84b8-416e44e4aef8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:51.476087 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49647e9e-a547-4e2f-84b8-416e44e4aef8 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:51.489152 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 481/976] 10.222.0.23 () {56 vars in 1249 bytes} [Tue Jun 21 18:06:51 2022] GET /identity/v3/endpoints?service_id=5e03d9146bcf463e9fcade22442438da&interface=admin®ion_id=RegionOne => generated 193 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:51.490925 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49647e9e-a547-4e2f-84b8-416e44e4aef8 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:51.627298 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b7d904-ddb0-47c5-b4cb-1591c00e92c7 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:51.660614 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:51.663056 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49647e9e-a547-4e2f-84b8-416e44e4aef8 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:51.679604 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817635d5-41ca-4553-ae63-da839abaeaaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:51.682056 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817635d5-41ca-4553-ae63-da839abaeaaf None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:51.682573 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817635d5-41ca-4553-ae63-da839abaeaaf None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:51.683191 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817635d5-41ca-4553-ae63-da839abaeaaf None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:51.708431 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817635d5-41ca-4553-ae63-da839abaeaaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:06:51.712935 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:51.716025 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-817635d5-41ca-4553-ae63-da839abaeaaf None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 18:06:51.716025 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:51.716025 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:51.716025 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:51.716025 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:51.716025 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:51.716025 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:51.716025 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:51.716025 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:51.716025 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:51.716025 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:51.716025 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:51.716025 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:06:51.716025 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:06:51.716025 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:06:51.716025 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:06:51.718162 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:06:51.718162 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:06:51.718162 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:06:51.718162 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:06:51.718162 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:51.718162 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:51.718162 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:51.718162 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:51.718162 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:51.718162 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:51.718162 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:51.718162 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:51.718162 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:51.718162 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:51.718162 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:51.718162 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:51.720052 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:51.720052 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:51.720052 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:51.720052 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:51.720052 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:06:51.720052 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:06:51.720052 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:06:51.720052 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:06:51.720052 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:06:51.720052 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:06:51.720052 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 18:06:51.720052 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:51.722403 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 496/977] 10.222.0.23 () {56 vars in 1120 bytes} [Tue Jun 21 18:06:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:51.723458 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817635d5-41ca-4553-ae63-da839abaeaaf None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:51.729769 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc110bc2-3289-4113-9583-7baca4731b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:51.730121 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc110bc2-3289-4113-9583-7baca4731b50 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:51.730602 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc110bc2-3289-4113-9583-7baca4731b50 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:51.731060 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc110bc2-3289-4113-9583-7baca4731b50 None admin] PATH_INFO: `/v3/services` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:51.732371 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc110bc2-3289-4113-9583-7baca4731b50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:51.735202 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc110bc2-3289-4113-9583-7baca4731b50 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:51.750786 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 482/978] 10.222.0.23 () {56 vars in 1087 bytes} [Tue Jun 21 18:06:51 2022] GET /identity/v3/services => generated 1727 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:06:51.860300 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817635d5-41ca-4553-ae63-da839abaeaaf None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:51.912038 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83207392-fdc9-42f7-9c80-a8eb316e35ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:51.912662 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83207392-fdc9-42f7-9c80-a8eb316e35ea None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:51.912941 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83207392-fdc9-42f7-9c80-a8eb316e35ea None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:51.913213 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83207392-fdc9-42f7-9c80-a8eb316e35ea None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:51.913991 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83207392-fdc9-42f7-9c80-a8eb316e35ea None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:51.914361 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83207392-fdc9-42f7-9c80-a8eb316e35ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:51.916198 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83207392-fdc9-42f7-9c80-a8eb316e35ea None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:51.926616 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 497/979] 10.222.0.23 () {56 vars in 1115 bytes} [Tue Jun 21 18:06:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:51.933180 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc110bc2-3289-4113-9583-7baca4731b50 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:52.082168 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc110bc2-3289-4113-9583-7baca4731b50 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:52.143594 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb1aecd-f7ef-408a-9e09-4839c6626332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:52.144672 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb1aecd-f7ef-408a-9e09-4839c6626332 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:52.145197 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb1aecd-f7ef-408a-9e09-4839c6626332 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:52.145586 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb1aecd-f7ef-408a-9e09-4839c6626332 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:52.159300 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb1aecd-f7ef-408a-9e09-4839c6626332 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:52.161076 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb1aecd-f7ef-408a-9e09-4839c6626332 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:52.183035 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-aeb1aecd-f7ef-408a-9e09-4839c6626332 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:52.183035 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:52.183035 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:52.183035 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:52.183035 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:52.183035 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:52.183035 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:52.183035 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:52.183035 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:52.183035 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:52.183035 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:52.183035 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:52.183035 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:06:52.183035 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:06:52.183035 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:06:52.184924 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:06:52.184924 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:52.184924 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:52.184924 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:52.184924 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:52.184924 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:52.184924 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:52.184924 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:52.184924 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:52.184924 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:52.184924 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:52.184924 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:52.184924 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:52.184924 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:52.184924 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:52.184924 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:52.184924 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:52.186288 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:06:52.186288 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:06:52.186288 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:52.186288 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:52.186288 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 483/980] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:06:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:06:52.193116 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83207392-fdc9-42f7-9c80-a8eb316e35ea None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:52.257924 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:52.326678 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83207392-fdc9-42f7-9c80-a8eb316e35ea None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:52.367490 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4174b4-eed2-4989-87c0-58b8a869ef8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:52.369446 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4174b4-eed2-4989-87c0-58b8a869ef8c None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:52.369638 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4174b4-eed2-4989-87c0-58b8a869ef8c None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:52.369912 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4174b4-eed2-4989-87c0-58b8a869ef8c None admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:52.370634 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4174b4-eed2-4989-87c0-58b8a869ef8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:52.370937 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4174b4-eed2-4989-87c0-58b8a869ef8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:52.372330 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4174b4-eed2-4989-87c0-58b8a869ef8c None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:52.386719 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 498/981] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:06:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:52.392834 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb1aecd-f7ef-408a-9e09-4839c6626332 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:52.462724 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:52.534451 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb1aecd-f7ef-408a-9e09-4839c6626332 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:52.583078 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f06aed-e1f2-4633-9de2-15ff4e450971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:52.584906 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f06aed-e1f2-4633-9de2-15ff4e450971 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:52.585604 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f06aed-e1f2-4633-9de2-15ff4e450971 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:52.586323 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f06aed-e1f2-4633-9de2-15ff4e450971 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:52.602276 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f06aed-e1f2-4633-9de2-15ff4e450971 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:52.605687 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f06aed-e1f2-4633-9de2-15ff4e450971 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:52.637988 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-16f06aed-e1f2-4633-9de2-15ff4e450971 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 18:06:52.637988 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:52.637988 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:52.637988 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:52.637988 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:52.637988 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:52.637988 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:52.637988 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:52.637988 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:52.637988 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:52.637988 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:52.637988 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:52.637988 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:06:52.637988 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:06:52.637988 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:06:52.637988 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:06:52.640023 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:52.640023 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:52.640023 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:06:52.640023 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:06:52.640023 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:06:52.640023 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:06:52.640023 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:52.640023 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:52.640023 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:52.640023 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:52.640023 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:52.640023 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:52.640023 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:52.640023 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:52.640023 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:52.640023 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:52.640023 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:52.640023 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:52.651131 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:52.651131 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:52.651131 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:06:52.651131 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:06:52.651131 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:06:52.651131 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:06:52.651131 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:06:52.651131 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:06:52.651131 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 18:06:52.651131 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:52.651131 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 484/982] 10.222.0.23 () {56 vars in 1117 bytes} [Tue Jun 21 18:06:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:06:52.657632 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4174b4-eed2-4989-87c0-58b8a869ef8c None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:52.784310 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4174b4-eed2-4989-87c0-58b8a869ef8c None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:52.793784 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:52.826542 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bad99a6-f019-425b-a298-06c8f78245f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:52.827244 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad99a6-f019-425b-a298-06c8f78245f6 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:52.827525 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad99a6-f019-425b-a298-06c8f78245f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:52.827800 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad99a6-f019-425b-a298-06c8f78245f6 None admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:52.828777 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bad99a6-f019-425b-a298-06c8f78245f6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:52.829142 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bad99a6-f019-425b-a298-06c8f78245f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:52.830576 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bad99a6-f019-425b-a298-06c8f78245f6 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:52.860248 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f06aed-e1f2-4633-9de2-15ff4e450971 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:52.861076 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 499/983] 10.222.0.23 () {56 vars in 1149 bytes} [Tue Jun 21 18:06:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:52.976834 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:52.981318 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f06aed-e1f2-4633-9de2-15ff4e450971 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:53.041078 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93553345-29ec-43fd-99a2-b0b74d08aa7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:53.042067 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93553345-29ec-43fd-99a2-b0b74d08aa7a None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:53.042542 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93553345-29ec-43fd-99a2-b0b74d08aa7a None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:53.042985 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93553345-29ec-43fd-99a2-b0b74d08aa7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:53.176672 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93553345-29ec-43fd-99a2-b0b74d08aa7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:53.179380 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93553345-29ec-43fd-99a2-b0b74d08aa7a None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:53.369756 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:53.378712 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 485/984] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:06:52 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 18:06:53.387422 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bad99a6-f019-425b-a298-06c8f78245f6 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:53.515770 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bad99a6-f019-425b-a298-06c8f78245f6 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:53.559476 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fea4b9c-ae93-40de-9f8c-eb153bf9121c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:53.560574 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea4b9c-ae93-40de-9f8c-eb153bf9121c None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:53.560996 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea4b9c-ae93-40de-9f8c-eb153bf9121c None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:53.561409 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea4b9c-ae93-40de-9f8c-eb153bf9121c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:53.574898 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fea4b9c-ae93-40de-9f8c-eb153bf9121c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:53.576730 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fea4b9c-ae93-40de-9f8c-eb153bf9121c None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:53.598519 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-0fea4b9c-ae93-40de-9f8c-eb153bf9121c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:53.598519 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:53.598519 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:53.598519 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:53.598519 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:53.598519 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:53.598519 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:53.598519 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:53.598519 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:53.598519 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:53.598519 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:53.598519 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:53.598519 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:06:53.598519 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:06:53.598519 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:06:53.601051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:06:53.601051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:53.601051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:53.601051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:53.601051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:53.601051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:53.601051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:53.601051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:53.601051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:53.601051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:53.601051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:53.601051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:53.601051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:53.601051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:53.601051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:53.601051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:53.601051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:53.603379 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:06:53.603379 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:06:53.603379 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:06:53.603379 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:53.604853 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 500/985] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:06:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:53.609582 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93553345-29ec-43fd-99a2-b0b74d08aa7a None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:53.648455 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7391e207-89b8-4b8c-a181-5fc34ecb3733 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:53.648928 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7391e207-89b8-4b8c-a181-5fc34ecb3733 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:53.649920 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7391e207-89b8-4b8c-a181-5fc34ecb3733 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:53.651647 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 501/986] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:06:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:06:53.657261 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe08f8f-9317-4f7b-8636-8bf8bf8893c2 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:53.657577 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe08f8f-9317-4f7b-8636-8bf8bf8893c2 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:53.657849 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe08f8f-9317-4f7b-8636-8bf8bf8893c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:53.675775 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-dfe08f8f-9317-4f7b-8636-8bf8bf8893c2 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:06:53.677026 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-dfe08f8f-9317-4f7b-8636-8bf8bf8893c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:06:53.689519 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:53.764792 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 502/987] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:06:53 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:06:53.775775 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93553345-29ec-43fd-99a2-b0b74d08aa7a None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:53.777677 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe08f8f-9317-4f7b-8636-8bf8bf8893c2 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:53.828027 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e0f622-cecb-4bf7-b8de-e399e952f3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:53.828891 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e0f622-cecb-4bf7-b8de-e399e952f3d4 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:53.829169 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e0f622-cecb-4bf7-b8de-e399e952f3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:53.829452 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e0f622-cecb-4bf7-b8de-e399e952f3d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:53.830200 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e0f622-cecb-4bf7-b8de-e399e952f3d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:53.830511 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e0f622-cecb-4bf7-b8de-e399e952f3d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:53.831907 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e0f622-cecb-4bf7-b8de-e399e952f3d4 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:53.852463 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 486/988] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:06:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:53.855971 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e0f622-cecb-4bf7-b8de-e399e952f3d4 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:53.928113 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe08f8f-9317-4f7b-8636-8bf8bf8893c2 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:53.937194 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:53.986214 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e0f622-cecb-4bf7-b8de-e399e952f3d4 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:53.995603 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b14168-fade-434f-92b8-182cdbb8a482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:53.997052 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b14168-fade-434f-92b8-182cdbb8a482 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:53.997563 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b14168-fade-434f-92b8-182cdbb8a482 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:53.998062 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b14168-fade-434f-92b8-182cdbb8a482 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:53.999332 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b14168-fade-434f-92b8-182cdbb8a482 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:54.002033 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b14168-fade-434f-92b8-182cdbb8a482 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:54.023915 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1537245-463d-4a97-82f2-1fad50502e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:54.025350 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1537245-463d-4a97-82f2-1fad50502e18 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:54.025633 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1537245-463d-4a97-82f2-1fad50502e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:54.025907 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1537245-463d-4a97-82f2-1fad50502e18 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:54.026282 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-40b14168-fade-434f-92b8-182cdbb8a482 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 21 18:06:54.026282 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:54.026282 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:06:54.026282 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:06:54.026282 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:06:54.026282 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:06:54.026282 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:06:54.026282 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:06:54.026282 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 21 18:06:54.026282 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:54.026282 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:06:54.026282 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:54.026282 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:54.026282 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:54.026282 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:54.026282 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:54.026282 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:54.027254 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:54.027254 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:54.027254 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:54.027254 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:54.027254 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:54.027254 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:54.027254 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:06:54.027254 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:06:54.027254 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:06:54.027254 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:06:54.027254 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:54.027254 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:54.027254 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:54.027254 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:54.027254 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:54.027254 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:54.028938 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:54.028938 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:54.028938 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:54.028938 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:54.028938 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:54.028938 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:54.028938 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:54.028938 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:54.028938 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:54.028938 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:54.028938 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:06:54.028938 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:06:54.028938 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 21 18:06:54.028938 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:54.028938 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 503/989] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jun 21 18:06:53 2022] GET /identity/v3/services/compute => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:06:54.037296 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b14168-fade-434f-92b8-182cdbb8a482 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:54.041847 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1537245-463d-4a97-82f2-1fad50502e18 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:54.043337 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1537245-463d-4a97-82f2-1fad50502e18 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:54.058936 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-a1537245-463d-4a97-82f2-1fad50502e18 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:06:54.058936 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:54.058936 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:54.058936 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:54.058936 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:54.058936 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:54.058936 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:54.058936 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:54.058936 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:54.058936 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:54.058936 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:54.058936 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:54.058936 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:06:54.058936 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:06:54.058936 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:06:54.060183 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:06:54.060183 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:54.060183 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:54.060183 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:54.060183 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:54.060183 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:54.060183 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:54.060183 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:54.060183 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:54.060183 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:54.060183 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:54.060183 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:54.060183 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:54.060183 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:54.060183 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:54.060183 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:54.060183 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:54.061736 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:06:54.061736 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:06:54.061736 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:06:54.061736 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:06:54.061736 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:06:54.061736 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:06:54.061736 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:06:54.061736 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:54.065219 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 487/990] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jun 21 18:06:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:54.065997 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1537245-463d-4a97-82f2-1fad50502e18 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:54.066595 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:54.205255 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b14168-fade-434f-92b8-182cdbb8a482 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:54.228959 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1537245-463d-4a97-82f2-1fad50502e18 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:54.261386 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:54.266065 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfb8a74-5b6d-4056-bb6a-9ee757187c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:54.267003 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfb8a74-5b6d-4056-bb6a-9ee757187c0a None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:54.267433 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfb8a74-5b6d-4056-bb6a-9ee757187c0a None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:54.267901 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfb8a74-5b6d-4056-bb6a-9ee757187c0a None admin] PATH_INFO: `/v3/services` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:54.269215 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfb8a74-5b6d-4056-bb6a-9ee757187c0a None admin] RBAC: Adding query filter params (name=compute) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:54.269733 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfb8a74-5b6d-4056-bb6a-9ee757187c0a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:54.272160 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfb8a74-5b6d-4056-bb6a-9ee757187c0a None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:54.282120 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 504/991] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Jun 21 18:06:54 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:54.287313 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfb8a74-5b6d-4056-bb6a-9ee757187c0a None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:54.295067 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc4d71c-1771-4b91-93da-fdeb69e3753c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:54.296129 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4d71c-1771-4b91-93da-fdeb69e3753c None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:54.296633 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4d71c-1771-4b91-93da-fdeb69e3753c None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:54.297170 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4d71c-1771-4b91-93da-fdeb69e3753c None admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:54.298568 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc4d71c-1771-4b91-93da-fdeb69e3753c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:54.299076 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc4d71c-1771-4b91-93da-fdeb69e3753c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:54.301693 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc4d71c-1771-4b91-93da-fdeb69e3753c None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:54.328314 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 488/992] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jun 21 18:06:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:06:54.329843 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc4d71c-1771-4b91-93da-fdeb69e3753c None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:54.418526 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:54.450821 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfb8a74-5b6d-4056-bb6a-9ee757187c0a None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:54.476481 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc4d71c-1771-4b91-93da-fdeb69e3753c None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:54.489806 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe49c112-d2dd-45d1-ba2f-bd1e50c4de0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:54.490487 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe49c112-d2dd-45d1-ba2f-bd1e50c4de0b None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:54.490790 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe49c112-d2dd-45d1-ba2f-bd1e50c4de0b None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:54.491062 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe49c112-d2dd-45d1-ba2f-bd1e50c4de0b None admin] PATH_INFO: `/v3/services` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:54.491790 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe49c112-d2dd-45d1-ba2f-bd1e50c4de0b None admin] RBAC: Adding query filter params (type=compute) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:54.492094 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe49c112-d2dd-45d1-ba2f-bd1e50c4de0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:54.493753 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe49c112-d2dd-45d1-ba2f-bd1e50c4de0b None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:54.499111 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 505/993] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Jun 21 18:06:54 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:54.500578 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10956fc7-f1e9-489d-ae7e-f70b92708baa None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:54.500872 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10956fc7-f1e9-489d-ae7e-f70b92708baa None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:54.501157 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10956fc7-f1e9-489d-ae7e-f70b92708baa None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:54.502683 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 506/994] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:06:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:06:54.506717 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10956fc7-f1e9-489d-ae7e-f70b92708baa None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:54.518272 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffce369-60ad-4cbd-a4ee-6edc0ac18c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:54.519343 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffce369-60ad-4cbd-a4ee-6edc0ac18c8f None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:54.519803 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffce369-60ad-4cbd-a4ee-6edc0ac18c8f None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:54.520264 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffce369-60ad-4cbd-a4ee-6edc0ac18c8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:54.521500 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffce369-60ad-4cbd-a4ee-6edc0ac18c8f None admin] RBAC: Adding query filter params (role.id=e950e7aaae294af383fc9e83eed8250f, scope.project.id=2a4699d1ec70456da5f8e0aa4b1439b7, user.id=fcfcd9f134e74b09bfdac337c81c8a07) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:54.522011 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffce369-60ad-4cbd-a4ee-6edc0ac18c8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:54.525119 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffce369-60ad-4cbd-a4ee-6edc0ac18c8f None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:54.540490 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 489/995] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jun 21 18:06:54 2022] GET /identity/v3/role_assignments?user.id=fcfcd9f134e74b09bfdac337c81c8a07&scope.project.id=2a4699d1ec70456da5f8e0aa4b1439b7&role.id=e950e7aaae294af383fc9e83eed8250f => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:06:54.540490 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612bc4f-a218-45da-a608-49abb53f1020 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:54.540490 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612bc4f-a218-45da-a608-49abb53f1020 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:54.540490 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612bc4f-a218-45da-a608-49abb53f1020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:54.567037 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-2612bc4f-a218-45da-a608-49abb53f1020 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:06:54.568339 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-2612bc4f-a218-45da-a608-49abb53f1020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:06:54.620551 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:54.663077 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10956fc7-f1e9-489d-ae7e-f70b92708baa None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:54.668222 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 490/996] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:06:54 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:06:54.692810 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2612bc4f-a218-45da-a608-49abb53f1020 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:54.721463 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb838f-506e-49a0-bfe4-b96a9b249cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:54.722562 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb838f-506e-49a0-bfe4-b96a9b249cce None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:54.723726 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb838f-506e-49a0-bfe4-b96a9b249cce None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:54.723726 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb838f-506e-49a0-bfe4-b96a9b249cce None admin] PATH_INFO: `/v3/endpoints` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:54.724974 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cb838f-506e-49a0-bfe4-b96a9b249cce None admin] RBAC: Adding query filter params (interface=public, service_id=1a4ca9ad8f164260a29a1d1b8f82d2cd, region_id=RegionOne) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:54.725501 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cb838f-506e-49a0-bfe4-b96a9b249cce None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:54.727789 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cb838f-506e-49a0-bfe4-b96a9b249cce None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:54.737423 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 507/997] 10.222.0.23 () {56 vars in 1251 bytes} [Tue Jun 21 18:06:54 2022] GET /identity/v3/endpoints?service_id=1a4ca9ad8f164260a29a1d1b8f82d2cd&interface=public®ion_id=RegionOne => generated 194 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:54.745293 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb838f-506e-49a0-bfe4-b96a9b249cce None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:54.749919 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:54.848732 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2612bc4f-a218-45da-a608-49abb53f1020 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:54.908623 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5864e62-ea1d-4183-a08e-2ebd383bf988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:54.910049 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5864e62-ea1d-4183-a08e-2ebd383bf988 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:54.910540 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5864e62-ea1d-4183-a08e-2ebd383bf988 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:54.911095 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5864e62-ea1d-4183-a08e-2ebd383bf988 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:54.912247 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5864e62-ea1d-4183-a08e-2ebd383bf988 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:54.915497 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5864e62-ea1d-4183-a08e-2ebd383bf988 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:54.916115 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb838f-506e-49a0-bfe4-b96a9b249cce None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:54.939823 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-b5864e62-ea1d-4183-a08e-2ebd383bf988 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 18:06:54.939823 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:54.939823 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:06:54.939823 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:06:54.939823 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:06:54.939823 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:06:54.939823 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:06:54.939823 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:06:54.939823 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 18:06:54.939823 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:54.939823 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:06:54.939823 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:54.939823 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:54.939823 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:54.939823 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:54.939823 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:54.939823 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:54.942416 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:54.942416 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:54.942416 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:54.942416 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:54.942416 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:54.942416 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:54.942416 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:06:54.942416 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:06:54.942416 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:06:54.942416 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:06:54.942416 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:54.942416 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:54.942416 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:54.942416 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:54.942416 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:54.942416 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:54.945559 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:54.945559 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:54.945559 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:54.945559 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:54.945559 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:54.945559 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:54.945559 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:54.945559 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:54.945559 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:54.945559 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:54.945559 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:06:54.945559 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:06:54.945559 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 18:06:54.945559 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:54.945559 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 491/998] 10.222.0.23 () {56 vars in 1126 bytes} [Tue Jun 21 18:06:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:06:54.949215 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5864e62-ea1d-4183-a08e-2ebd383bf988 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:54.966393 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edcfbc3-c4f7-45b4-8c5f-7d0569f443c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:54.967211 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edcfbc3-c4f7-45b4-8c5f-7d0569f443c1 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:54.967541 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edcfbc3-c4f7-45b4-8c5f-7d0569f443c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:54.967888 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edcfbc3-c4f7-45b4-8c5f-7d0569f443c1 None admin] PATH_INFO: `/v3/services` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:54.968775 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edcfbc3-c4f7-45b4-8c5f-7d0569f443c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:54.970348 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edcfbc3-c4f7-45b4-8c5f-7d0569f443c1 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:54.980035 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 508/999] 10.222.0.23 () {56 vars in 1087 bytes} [Tue Jun 21 18:06:54 2022] GET /identity/v3/services => generated 1727 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:06:55.023982 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:55.163867 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5864e62-ea1d-4183-a08e-2ebd383bf988 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:55.221004 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cf585d-604f-4dee-97aa-f89b45e90ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:55.222124 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf585d-604f-4dee-97aa-f89b45e90ada None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:55.222613 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf585d-604f-4dee-97aa-f89b45e90ada None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:55.223111 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf585d-604f-4dee-97aa-f89b45e90ada None admin] PATH_INFO: `/v3/services` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:55.224736 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cf585d-604f-4dee-97aa-f89b45e90ada None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:55.225301 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cf585d-604f-4dee-97aa-f89b45e90ada None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:55.228381 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cf585d-604f-4dee-97aa-f89b45e90ada None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:55.238399 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 492/1000] 10.222.0.23 () {56 vars in 1122 bytes} [Tue Jun 21 18:06:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:55.244796 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edcfbc3-c4f7-45b4-8c5f-7d0569f443c1 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:55.342842 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:55.411411 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edcfbc3-c4f7-45b4-8c5f-7d0569f443c1 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:55.481027 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49a2b67-4133-4eb3-b742-d579dbb89f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:55.482244 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a2b67-4133-4eb3-b742-d579dbb89f2b None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:55.482732 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a2b67-4133-4eb3-b742-d579dbb89f2b None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:55.483227 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a2b67-4133-4eb3-b742-d579dbb89f2b None admin] PATH_INFO: `/v3/services` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:55.484556 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49a2b67-4133-4eb3-b742-d579dbb89f2b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:55.485047 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49a2b67-4133-4eb3-b742-d579dbb89f2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:55.487507 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49a2b67-4133-4eb3-b742-d579dbb89f2b None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:55.503016 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 509/1001] 10.222.0.23 () {56 vars in 1122 bytes} [Tue Jun 21 18:06:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:06:55.504751 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cf585d-604f-4dee-97aa-f89b45e90ada None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:55.671077 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:55.676028 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cf585d-604f-4dee-97aa-f89b45e90ada None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:55.734304 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6535e7ec-bdb5-478c-b56a-343d07f4ebbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:55.735339 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6535e7ec-bdb5-478c-b56a-343d07f4ebbb None admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:55.735740 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6535e7ec-bdb5-478c-b56a-343d07f4ebbb None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:55.736118 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6535e7ec-bdb5-478c-b56a-343d07f4ebbb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:55.737256 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6535e7ec-bdb5-478c-b56a-343d07f4ebbb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:55.739293 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6535e7ec-bdb5-478c-b56a-343d07f4ebbb None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:55.820288 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 493/1002] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Jun 21 18:06:55 2022] POST /identity/v3/endpoints => generated 325 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:06:56.979846 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7cba32-54fd-4947-b6bc-e40c2e862b76 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:56.980729 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7cba32-54fd-4947-b6bc-e40c2e862b76 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:56.980729 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7cba32-54fd-4947-b6bc-e40c2e862b76 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:56.984430 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 510/1003] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:06:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:06:56.990501 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ecefa-3dc3-4a88-b346-75c9a3e8e6a9 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:56.991036 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ecefa-3dc3-4a88-b346-75c9a3e8e6a9 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:56.991449 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ecefa-3dc3-4a88-b346-75c9a3e8e6a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:57.019925 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-436ecefa-3dc3-4a88-b346-75c9a3e8e6a9 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:06:57.021924 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-436ecefa-3dc3-4a88-b346-75c9a3e8e6a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:06:57.185483 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 494/1004] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:06:56 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:06:57.209070 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7cba32-54fd-4947-b6bc-e40c2e862b76 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:57.346037 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7cba32-54fd-4947-b6bc-e40c2e862b76 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:57.388462 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:57.396626 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ebfaf6-c167-4b99-b404-245dbe0be977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:57.397754 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ebfaf6-c167-4b99-b404-245dbe0be977 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:57.398145 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ebfaf6-c167-4b99-b404-245dbe0be977 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:57.398524 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ebfaf6-c167-4b99-b404-245dbe0be977 None admin] PATH_INFO: `/v3/services/image` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:57.400069 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ebfaf6-c167-4b99-b404-245dbe0be977 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:57.402146 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ebfaf6-c167-4b99-b404-245dbe0be977 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:57.421477 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-46ebfaf6-c167-4b99-b404-245dbe0be977 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 21 18:06:57.421477 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:57.421477 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:06:57.421477 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:06:57.421477 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:06:57.421477 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:06:57.421477 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:06:57.421477 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:06:57.421477 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 21 18:06:57.421477 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:57.421477 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:06:57.421477 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:57.421477 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:57.421477 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:57.421477 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:57.421477 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:57.421477 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:57.422602 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:57.422602 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:57.422602 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:57.422602 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:57.422602 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:57.422602 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:57.422602 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:06:57.422602 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:06:57.422602 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:06:57.422602 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:06:57.422602 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:57.422602 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:57.422602 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:57.422602 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:57.422602 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:57.422602 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:57.428498 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:57.428498 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:57.428498 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:57.428498 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:57.428498 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:57.428498 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:57.428498 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:57.428498 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:57.428498 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:57.428498 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:57.428498 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:06:57.428498 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:06:57.428498 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 21 18:06:57.428498 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:06:57.428498 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 511/1005] 10.222.0.23 () {56 vars in 1105 bytes} [Tue Jun 21 18:06:57 2022] GET /identity/v3/services/image => generated 86 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:06:57.432249 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436ecefa-3dc3-4a88-b346-75c9a3e8e6a9 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:57.539372 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:57.558340 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436ecefa-3dc3-4a88-b346-75c9a3e8e6a9 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:57.595665 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba82bdc-fe5e-433c-a889-ac4a98eabb72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:57.598708 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba82bdc-fe5e-433c-a889-ac4a98eabb72 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:57.598708 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba82bdc-fe5e-433c-a889-ac4a98eabb72 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:57.598708 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba82bdc-fe5e-433c-a889-ac4a98eabb72 None admin] PATH_INFO: `/v3/services` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:57.598708 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba82bdc-fe5e-433c-a889-ac4a98eabb72 None admin] RBAC: Adding query filter params (name=image) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:57.598708 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba82bdc-fe5e-433c-a889-ac4a98eabb72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:57.599613 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba82bdc-fe5e-433c-a889-ac4a98eabb72 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:57.610991 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 495/1006] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:06:57 2022] GET /identity/v3/services?name=image => generated 122 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:57.612345 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ebfaf6-c167-4b99-b404-245dbe0be977 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:57.735708 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ebfaf6-c167-4b99-b404-245dbe0be977 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:57.776134 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71040163-3ad1-4f46-8591-f042a8dbfae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:57.777147 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71040163-3ad1-4f46-8591-f042a8dbfae0 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:57.777617 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71040163-3ad1-4f46-8591-f042a8dbfae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:57.778110 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71040163-3ad1-4f46-8591-f042a8dbfae0 None admin] PATH_INFO: `/v3/services` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:57.779225 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71040163-3ad1-4f46-8591-f042a8dbfae0 None admin] RBAC: Adding query filter params (type=image) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:57.779849 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71040163-3ad1-4f46-8591-f042a8dbfae0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:57.781475 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71040163-3ad1-4f46-8591-f042a8dbfae0 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:57.787483 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 512/1007] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:06:57 2022] GET /identity/v3/services?type=image => generated 122 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:58.059519 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab2ade-740e-4847-bc7d-b9607b26e07c None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:58.060327 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab2ade-740e-4847-bc7d-b9607b26e07c None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:58.060656 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab2ade-740e-4847-bc7d-b9607b26e07c None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:58.062469 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 496/1008] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:06:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:06:58.068255 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c01dc0-37dd-4d9e-86e8-0b0430aaf373 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:58.069102 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c01dc0-37dd-4d9e-86e8-0b0430aaf373 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:58.069246 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c01dc0-37dd-4d9e-86e8-0b0430aaf373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:58.089530 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-88c01dc0-37dd-4d9e-86e8-0b0430aaf373 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:06:58.090944 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-88c01dc0-37dd-4d9e-86e8-0b0430aaf373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:06:58.167787 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:58.188514 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 513/1009] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:06:58 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:06:58.209611 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ab2ade-740e-4847-bc7d-b9607b26e07c None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:58.393442 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ab2ade-740e-4847-bc7d-b9607b26e07c None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:58.458249 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e40fe6-fb25-4ef1-801e-6a599a2d0899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:58.459524 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e40fe6-fb25-4ef1-801e-6a599a2d0899 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:58.460022 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e40fe6-fb25-4ef1-801e-6a599a2d0899 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:58.460538 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e40fe6-fb25-4ef1-801e-6a599a2d0899 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:58.462293 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e40fe6-fb25-4ef1-801e-6a599a2d0899 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:58.464951 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e40fe6-fb25-4ef1-801e-6a599a2d0899 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:58.488206 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-75e40fe6-fb25-4ef1-801e-6a599a2d0899 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 21 18:06:58.488206 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:58.488206 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:06:58.488206 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:06:58.488206 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:06:58.488206 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:06:58.488206 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:06:58.488206 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:06:58.488206 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 21 18:06:58.488206 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:58.488206 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:06:58.488206 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:58.488206 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:06:58.488206 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:06:58.488206 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:06:58.488206 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:06:58.488206 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:06:58.489357 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:06:58.489357 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:06:58.489357 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:06:58.489357 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:06:58.489357 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:06:58.489357 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:06:58.489357 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:06:58.489357 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:06:58.489357 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:06:58.489357 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:06:58.489357 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:06:58.489357 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:06:58.489357 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:06:58.489357 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:06:58.489357 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:06:58.489357 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:06:58.490239 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:06:58.490239 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:06:58.490239 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:06:58.490239 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:06:58.490239 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:06:58.490239 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:06:58.490239 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:06:58.490239 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:06:58.490239 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:06:58.490239 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:06:58.490239 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:06:58.490239 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:06:58.490239 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 21 18:06:58.490239 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:06:58.497106 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 497/1010] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jun 21 18:06:58 2022] GET /identity/v3/services/compute => generated 88 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:06:58.498626 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c01dc0-37dd-4d9e-86e8-0b0430aaf373 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:58.645936 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e4d818-256f-446b-a3a7-83069072dd65 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:58.646477 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e4d818-256f-446b-a3a7-83069072dd65 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:58.646832 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e4d818-256f-446b-a3a7-83069072dd65 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:58.649362 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 498/1011] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:06:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:06:58.658261 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2f25e5-fb25-4d57-a46e-7815396f6d02 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:58.658721 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2f25e5-fb25-4d57-a46e-7815396f6d02 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:58.659174 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2f25e5-fb25-4d57-a46e-7815396f6d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:58.686030 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-cc2f25e5-fb25-4d57-a46e-7815396f6d02 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:06:58.686459 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c01dc0-37dd-4d9e-86e8-0b0430aaf373 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:58.687994 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-cc2f25e5-fb25-4d57-a46e-7815396f6d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:06:58.706816 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:58.762356 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b54e41b-be1a-4b0c-96e5-fafca3b3cca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:58.763533 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54e41b-be1a-4b0c-96e5-fafca3b3cca4 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:58.764023 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54e41b-be1a-4b0c-96e5-fafca3b3cca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:58.764621 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54e41b-be1a-4b0c-96e5-fafca3b3cca4 None admin] PATH_INFO: `/v3/services` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:58.765900 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b54e41b-be1a-4b0c-96e5-fafca3b3cca4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:58.766454 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b54e41b-be1a-4b0c-96e5-fafca3b3cca4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:58.769254 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b54e41b-be1a-4b0c-96e5-fafca3b3cca4 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:58.783664 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 514/1012] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Jun 21 18:06:58 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:58.794195 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b54e41b-be1a-4b0c-96e5-fafca3b3cca4 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:58.845253 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 499/1013] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:06:58 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:06:58.896133 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2f25e5-fb25-4d57-a46e-7815396f6d02 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:58.910192 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:59.011329 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b54e41b-be1a-4b0c-96e5-fafca3b3cca4 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:59.080706 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e86e4b5-8055-4420-9e75-09555b6c5e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:59.081678 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e86e4b5-8055-4420-9e75-09555b6c5e24 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:59.082075 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e86e4b5-8055-4420-9e75-09555b6c5e24 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:59.082385 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e86e4b5-8055-4420-9e75-09555b6c5e24 None admin] PATH_INFO: `/v3/services` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:59.083482 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e86e4b5-8055-4420-9e75-09555b6c5e24 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:06:59.083929 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e86e4b5-8055-4420-9e75-09555b6c5e24 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:59.086075 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e86e4b5-8055-4420-9e75-09555b6c5e24 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:59.096547 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 515/1014] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Jun 21 18:06:58 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:06:59.107259 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e86e4b5-8055-4420-9e75-09555b6c5e24 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:59.109408 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2f25e5-fb25-4d57-a46e-7815396f6d02 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:59.184554 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b61bcf-100e-4bfd-a660-c61db96fca02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:59.186303 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b61bcf-100e-4bfd-a660-c61db96fca02 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:59.186805 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b61bcf-100e-4bfd-a660-c61db96fca02 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:59.187316 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b61bcf-100e-4bfd-a660-c61db96fca02 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:59.189657 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b61bcf-100e-4bfd-a660-c61db96fca02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:59.192448 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b61bcf-100e-4bfd-a660-c61db96fca02 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:59.196515 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 500/1015] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:06:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:06:59.203461 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b61bcf-100e-4bfd-a660-c61db96fca02 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:06:59.226043 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:59.321604 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e86e4b5-8055-4420-9e75-09555b6c5e24 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:59.342593 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:06:59.365242 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b61bcf-100e-4bfd-a660-c61db96fca02 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:06:59.376692 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd6b0d8-161e-4bd1-8256-5a99cf613ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:59.377816 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd6b0d8-161e-4bd1-8256-5a99cf613ea5 None admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:59.378272 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd6b0d8-161e-4bd1-8256-5a99cf613ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:59.378694 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd6b0d8-161e-4bd1-8256-5a99cf613ea5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:59.380095 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd6b0d8-161e-4bd1-8256-5a99cf613ea5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:59.382698 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd6b0d8-161e-4bd1-8256-5a99cf613ea5 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:59.422598 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e9f190-fe9f-4f0f-9bfa-2e86648567ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:06:59.423807 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e9f190-fe9f-4f0f-9bfa-2e86648567ee None admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:06:59.424298 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e9f190-fe9f-4f0f-9bfa-2e86648567ee None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:06:59.424829 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e9f190-fe9f-4f0f-9bfa-2e86648567ee None admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:06:59.426164 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e9f190-fe9f-4f0f-9bfa-2e86648567ee None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:06:59.428900 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e9f190-fe9f-4f0f-9bfa-2e86648567ee None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:06:59.492444 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 516/1016] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Jun 21 18:06:59 2022] POST /identity/v3/endpoints => generated 334 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:06:59.572173 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 501/1017] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Jun 21 18:06:59 2022] POST /identity/v3/projects => generated 311 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:07:00.639016 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c71866-2c38-48c4-a621-dd93e4e2781a None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:00.639601 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c71866-2c38-48c4-a621-dd93e4e2781a None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:00.640114 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c71866-2c38-48c4-a621-dd93e4e2781a None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:00.642660 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 517/1018] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:07:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:07:00.648666 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af38e2a-d713-4f7e-8793-307e43afeb79 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:00.650565 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af38e2a-d713-4f7e-8793-307e43afeb79 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:00.650565 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af38e2a-d713-4f7e-8793-307e43afeb79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:00.668839 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-7af38e2a-d713-4f7e-8793-307e43afeb79 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:07:00.670064 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-7af38e2a-d713-4f7e-8793-307e43afeb79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:07:00.741778 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:00.769717 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 502/1019] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:07:00 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:07:00.793712 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c71866-2c38-48c4-a621-dd93e4e2781a None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:00.912821 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c71866-2c38-48c4-a621-dd93e4e2781a None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:00.968803 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b79fad5-21c9-4510-bc56-da0efc78075c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:00.969742 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79fad5-21c9-4510-bc56-da0efc78075c None admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:00.970138 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79fad5-21c9-4510-bc56-da0efc78075c None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:00.970517 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79fad5-21c9-4510-bc56-da0efc78075c None admin] PATH_INFO: `/v3/services` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:00.971600 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b79fad5-21c9-4510-bc56-da0efc78075c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:00.973729 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b79fad5-21c9-4510-bc56-da0efc78075c None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:01.109718 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 518/1020] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Jun 21 18:07:00 2022] POST /identity/v3/services => generated 242 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:07:02.042490 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6571d7-3587-4e42-bbd0-3cce09fa87fa None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:02.043152 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6571d7-3587-4e42-bbd0-3cce09fa87fa None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:02.043152 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6571d7-3587-4e42-bbd0-3cce09fa87fa None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:02.044747 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 503/1021] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:07:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:07:02.050371 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa0194-5a69-49d2-8d23-884c818ec202 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:02.051162 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa0194-5a69-49d2-8d23-884c818ec202 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:02.051162 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa0194-5a69-49d2-8d23-884c818ec202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:02.069510 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-33fa0194-5a69-49d2-8d23-884c818ec202 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:07:02.070726 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-33fa0194-5a69-49d2-8d23-884c818ec202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:07:02.137784 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:02.217449 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 519/1022] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:07:02 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:07:02.246468 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6571d7-3587-4e42-bbd0-3cce09fa87fa None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:02.263335 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e6e0a-d969-48d4-b98a-5bca68e99583 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:02.263838 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e6e0a-d969-48d4-b98a-5bca68e99583 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:02.264280 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e6e0a-d969-48d4-b98a-5bca68e99583 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:02.266999 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 520/1023] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:07:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:07:02.276130 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ef443-1d62-4699-9602-3c8906902ee0 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:02.276533 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ef443-1d62-4699-9602-3c8906902ee0 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:02.276728 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ef443-1d62-4699-9602-3c8906902ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:02.304647 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-478ef443-1d62-4699-9602-3c8906902ee0 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:07:02.306955 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-478ef443-1d62-4699-9602-3c8906902ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:07:02.406773 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 521/1024] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:07:02 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:07:02.437273 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478ef443-1d62-4699-9602-3c8906902ee0 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:02.447257 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6571d7-3587-4e42-bbd0-3cce09fa87fa None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:02.513988 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4eb5ed-c870-4923-bbb4-5375b76ca5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:02.515238 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4eb5ed-c870-4923-bbb4-5375b76ca5c6 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:02.515725 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4eb5ed-c870-4923-bbb4-5375b76ca5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:02.516220 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4eb5ed-c870-4923-bbb4-5375b76ca5c6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:02.518744 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4eb5ed-c870-4923-bbb4-5375b76ca5c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:02.521531 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4eb5ed-c870-4923-bbb4-5375b76ca5c6 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:02.525216 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 504/1025] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:07:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:07:02.534559 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4eb5ed-c870-4923-bbb4-5375b76ca5c6 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:02.546516 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:02.564941 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478ef443-1d62-4699-9602-3c8906902ee0 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:02.623150 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f028f7-a0b0-4969-b1db-6e391001ea39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:02.624040 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f028f7-a0b0-4969-b1db-6e391001ea39 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:02.624367 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f028f7-a0b0-4969-b1db-6e391001ea39 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:02.624685 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f028f7-a0b0-4969-b1db-6e391001ea39 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:02.630024 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:02.638793 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f028f7-a0b0-4969-b1db-6e391001ea39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:07:02.647249 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-70f028f7-a0b0-4969-b1db-6e391001ea39 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 18:07:02.647249 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:07:02.647249 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:07:02.647249 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:07:02.647249 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:07:02.647249 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:07:02.647249 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:07:02.647249 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:07:02.647249 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:07:02.647249 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:07:02.647249 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:07:02.647249 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:07:02.647249 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:07:02.647249 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:07:02.647249 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:07:02.647249 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:07:02.651174 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:07:02.651174 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:07:02.651174 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:07:02.651174 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:07:02.651174 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:07:02.651174 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:07:02.651174 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:07:02.651174 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:07:02.651174 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:07:02.651174 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:07:02.651174 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:07:02.651174 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:07:02.651174 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:07:02.651174 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:07:02.651174 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:07:02.651174 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:07:02.652739 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:07:02.652739 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:07:02.652739 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:07:02.652739 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:07:02.652739 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:07:02.652739 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:07:02.652739 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:07:02.652739 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:07:02.652739 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:07:02.652739 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:07:02.652739 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 18:07:02.652739 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:07:02.652739 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 522/1026] 10.222.0.23 () {56 vars in 1120 bytes} [Tue Jun 21 18:07:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:07:02.656142 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f028f7-a0b0-4969-b1db-6e391001ea39 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:02.697974 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4eb5ed-c870-4923-bbb4-5375b76ca5c6 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:02.744486 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b2614d-38c1-4490-b3e2-cfb11dd73eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:02.745556 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b2614d-38c1-4490-b3e2-cfb11dd73eb1 None admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:02.746035 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b2614d-38c1-4490-b3e2-cfb11dd73eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:02.746506 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b2614d-38c1-4490-b3e2-cfb11dd73eb1 None admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:02.747742 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b2614d-38c1-4490-b3e2-cfb11dd73eb1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:02.750237 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b2614d-38c1-4490-b3e2-cfb11dd73eb1 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:02.850757 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f028f7-a0b0-4969-b1db-6e391001ea39 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:02.873769 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 505/1027] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Jun 21 18:07:02 2022] POST /identity/v3/users => generated 285 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:07:02.914530 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13987d65-6c9d-48be-b87e-9a889a81631e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:02.915291 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13987d65-6c9d-48be-b87e-9a889a81631e None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:02.915660 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13987d65-6c9d-48be-b87e-9a889a81631e None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:02.915944 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13987d65-6c9d-48be-b87e-9a889a81631e None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:02.916829 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13987d65-6c9d-48be-b87e-9a889a81631e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:07:02.917192 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13987d65-6c9d-48be-b87e-9a889a81631e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:02.918789 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13987d65-6c9d-48be-b87e-9a889a81631e None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:02.929480 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 523/1028] 10.222.0.23 () {56 vars in 1115 bytes} [Tue Jun 21 18:07:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:07:02.936612 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b2614d-38c1-4490-b3e2-cfb11dd73eb1 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:03.069724 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b2614d-38c1-4490-b3e2-cfb11dd73eb1 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:03.089623 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:03.123510 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23c31eb-db93-4198-a692-ddcd3cfa3c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:03.124233 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23c31eb-db93-4198-a692-ddcd3cfa3c31 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:03.124545 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23c31eb-db93-4198-a692-ddcd3cfa3c31 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:03.124831 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23c31eb-db93-4198-a692-ddcd3cfa3c31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:03.139418 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23c31eb-db93-4198-a692-ddcd3cfa3c31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:03.141394 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23c31eb-db93-4198-a692-ddcd3cfa3c31 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:03.172316 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-d23c31eb-db93-4198-a692-ddcd3cfa3c31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:07:03.172316 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:07:03.172316 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:07:03.172316 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:07:03.172316 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:07:03.172316 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:07:03.172316 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:07:03.172316 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:07:03.172316 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:07:03.172316 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:07:03.172316 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:07:03.172316 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:07:03.172316 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:07:03.172316 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:07:03.172316 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:07:03.174426 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:07:03.174426 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:07:03.174426 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:07:03.174426 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:07:03.174426 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:07:03.174426 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:07:03.174426 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:07:03.174426 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:07:03.174426 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:07:03.174426 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:07:03.174426 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:07:03.174426 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:07:03.174426 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:07:03.174426 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:07:03.174426 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:07:03.174426 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:07:03.174426 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:07:03.176781 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:07:03.176781 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:07:03.176781 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:07:03.176781 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:07:03.176781 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 506/1029] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:07:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:07:03.184437 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13987d65-6c9d-48be-b87e-9a889a81631e None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:03.210335 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:03.375980 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13987d65-6c9d-48be-b87e-9a889a81631e None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:03.450305 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862d6500-9a3a-4260-9f77-fdda82a91f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:03.452065 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862d6500-9a3a-4260-9f77-fdda82a91f36 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:03.452939 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862d6500-9a3a-4260-9f77-fdda82a91f36 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:03.453773 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862d6500-9a3a-4260-9f77-fdda82a91f36 None admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:03.455542 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862d6500-9a3a-4260-9f77-fdda82a91f36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:07:03.456562 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862d6500-9a3a-4260-9f77-fdda82a91f36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:03.459370 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862d6500-9a3a-4260-9f77-fdda82a91f36 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:03.486149 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 524/1030] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:07:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:07:03.494709 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23c31eb-db93-4198-a692-ddcd3cfa3c31 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:03.649845 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b14fd-1dde-4512-9ec4-5cdcb49471f8 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:03.650419 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b14fd-1dde-4512-9ec4-5cdcb49471f8 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:03.651030 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b14fd-1dde-4512-9ec4-5cdcb49471f8 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:03.655563 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 525/1031] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:07:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:07:03.662492 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb41123f-811c-41c7-984c-87ebcc1e13ba None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:03.662958 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb41123f-811c-41c7-984c-87ebcc1e13ba None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:03.663525 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb41123f-811c-41c7-984c-87ebcc1e13ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:03.692134 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-eb41123f-811c-41c7-984c-87ebcc1e13ba None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:07:03.693980 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-eb41123f-811c-41c7-984c-87ebcc1e13ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:07:03.694588 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23c31eb-db93-4198-a692-ddcd3cfa3c31 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:03.759165 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209e0579-d4ab-4202-826b-300e3316ff30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:03.760570 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209e0579-d4ab-4202-826b-300e3316ff30 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:03.761064 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209e0579-d4ab-4202-826b-300e3316ff30 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:03.761544 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209e0579-d4ab-4202-826b-300e3316ff30 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:03.776730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209e0579-d4ab-4202-826b-300e3316ff30 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:03.779469 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209e0579-d4ab-4202-826b-300e3316ff30 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:03.783168 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:03.791279 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 526/1032] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:07:03 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:07:03.808677 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-209e0579-d4ab-4202-826b-300e3316ff30 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 21 18:07:03.808677 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:07:03.808677 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:07:03.808677 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:07:03.808677 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:07:03.808677 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:07:03.808677 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:07:03.808677 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:07:03.808677 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:07:03.808677 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:07:03.808677 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:07:03.808677 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:07:03.808677 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:07:03.808677 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:07:03.808677 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:07:03.808677 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:07:03.810598 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:07:03.810598 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:07:03.810598 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:07:03.810598 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:07:03.810598 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:07:03.810598 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:07:03.810598 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:07:03.810598 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:07:03.810598 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:07:03.810598 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:07:03.810598 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:07:03.810598 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:07:03.810598 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:07:03.810598 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:07:03.810598 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:07:03.810598 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:07:03.810598 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:07:03.810598 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:07:03.813508 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:07:03.813508 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:07:03.813508 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:07:03.813508 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:07:03.813508 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:07:03.813508 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:07:03.813508 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:07:03.813508 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:07:03.813508 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 21 18:07:03.813508 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:07:03.815745 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb41123f-811c-41c7-984c-87ebcc1e13ba None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:03.823601 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 507/1033] 10.222.0.23 () {56 vars in 1093 bytes} [Tue Jun 21 18:07:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:07:03.824886 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209e0579-d4ab-4202-826b-300e3316ff30 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:03.881532 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:03.969299 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb41123f-811c-41c7-984c-87ebcc1e13ba None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:04.000692 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209e0579-d4ab-4202-826b-300e3316ff30 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:04.028988 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcc58c8-d8dc-4a69-b1ec-48387bc8f0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:04.029983 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc58c8-d8dc-4a69-b1ec-48387bc8f0d0 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:04.030397 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc58c8-d8dc-4a69-b1ec-48387bc8f0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:04.030862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc58c8-d8dc-4a69-b1ec-48387bc8f0d0 None admin] PATH_INFO: `/v3/services/image` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:04.031938 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcc58c8-d8dc-4a69-b1ec-48387bc8f0d0 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:04.033555 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcc58c8-d8dc-4a69-b1ec-48387bc8f0d0 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:04.055677 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-5fcc58c8-d8dc-4a69-b1ec-48387bc8f0d0 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 21 18:07:04.055677 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:07:04.055677 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:07:04.055677 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:07:04.055677 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:07:04.055677 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:07:04.055677 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:07:04.055677 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:07:04.055677 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 21 18:07:04.055677 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:07:04.055677 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:07:04.055677 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:07:04.055677 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:07:04.055677 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:07:04.055677 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:07:04.055677 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:07:04.055677 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:07:04.058380 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:07:04.058380 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:07:04.058380 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:07:04.058380 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:07:04.058380 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:07:04.058380 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:07:04.058380 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:07:04.058380 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:07:04.058380 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:07:04.058380 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:07:04.058380 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:07:04.058380 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:07:04.058380 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:07:04.058380 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:07:04.058380 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:07:04.058380 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:07:04.060609 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:07:04.060609 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:07:04.060609 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:07:04.060609 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:07:04.060609 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:07:04.060609 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:07:04.060609 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:07:04.060609 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:07:04.060609 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:07:04.060609 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:07:04.060609 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:07:04.060609 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:07:04.060609 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 21 18:07:04.060609 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:07:04.060609 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 527/1034] 10.222.0.23 () {56 vars in 1105 bytes} [Tue Jun 21 18:07:03 2022] GET /identity/v3/services/image => generated 86 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:07:04.064264 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4613bd6-009e-46be-999f-6919e104ecee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:04.064898 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcc58c8-d8dc-4a69-b1ec-48387bc8f0d0 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:04.065280 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4613bd6-009e-46be-999f-6919e104ecee None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:04.065821 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4613bd6-009e-46be-999f-6919e104ecee None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:04.066230 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4613bd6-009e-46be-999f-6919e104ecee None admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:04.067251 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4613bd6-009e-46be-999f-6919e104ecee None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:07:04.067693 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4613bd6-009e-46be-999f-6919e104ecee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:04.069784 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4613bd6-009e-46be-999f-6919e104ecee None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:04.109020 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 508/1035] 10.222.0.23 () {56 vars in 1133 bytes} [Tue Jun 21 18:07:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:07:04.112655 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4613bd6-009e-46be-999f-6919e104ecee None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:04.183841 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:04.283610 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcc58c8-d8dc-4a69-b1ec-48387bc8f0d0 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:04.284434 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:04.295499 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4613bd6-009e-46be-999f-6919e104ecee None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:04.324335 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6821f747-6470-4e26-93c0-04d0e5cca400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:04.325528 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6821f747-6470-4e26-93c0-04d0e5cca400 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:04.326009 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6821f747-6470-4e26-93c0-04d0e5cca400 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:04.326437 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6821f747-6470-4e26-93c0-04d0e5cca400 None admin] PATH_INFO: `/v3/services` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:04.327673 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6821f747-6470-4e26-93c0-04d0e5cca400 None admin] RBAC: Adding query filter params (name=image) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:07:04.328643 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6821f747-6470-4e26-93c0-04d0e5cca400 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:04.330967 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6821f747-6470-4e26-93c0-04d0e5cca400 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:04.341343 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 528/1036] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:07:04 2022] GET /identity/v3/services?name=image => generated 122 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:07:04.347958 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59567c40-dadc-401c-9fed-6d2d0c64918b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:04.348962 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59567c40-dadc-401c-9fed-6d2d0c64918b None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:04.348962 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59567c40-dadc-401c-9fed-6d2d0c64918b None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:04.349527 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59567c40-dadc-401c-9fed-6d2d0c64918b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:04.351095 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6821f747-6470-4e26-93c0-04d0e5cca400 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:04.441475 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59567c40-dadc-401c-9fed-6d2d0c64918b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:04.444349 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59567c40-dadc-401c-9fed-6d2d0c64918b None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:04.510652 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6821f747-6470-4e26-93c0-04d0e5cca400 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:04.550638 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:04.564286 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a524083-7363-4cc7-a90c-68564b9ff194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:04.565364 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a524083-7363-4cc7-a90c-68564b9ff194 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:04.565648 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a524083-7363-4cc7-a90c-68564b9ff194 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:04.565931 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a524083-7363-4cc7-a90c-68564b9ff194 None admin] PATH_INFO: `/v3/services` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:04.566657 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a524083-7363-4cc7-a90c-68564b9ff194 None admin] RBAC: Adding query filter params (type=image) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:07:04.566963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a524083-7363-4cc7-a90c-68564b9ff194 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:04.568684 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a524083-7363-4cc7-a90c-68564b9ff194 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:04.574122 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 529/1037] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:07:04 2022] GET /identity/v3/services?type=image => generated 350 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:07:04.580086 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a524083-7363-4cc7-a90c-68564b9ff194 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:04.581295 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:04.613308 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 509/1038] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:07:04 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 21 18:07:04.619845 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59567c40-dadc-401c-9fed-6d2d0c64918b None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:04.704281 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a524083-7363-4cc7-a90c-68564b9ff194 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:04.746098 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3d7d27-2ade-40b2-8e78-fb8fd606a324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:04.746784 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d7d27-2ade-40b2-8e78-fb8fd606a324 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:04.747072 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d7d27-2ade-40b2-8e78-fb8fd606a324 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:04.747356 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d7d27-2ade-40b2-8e78-fb8fd606a324 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:04.748179 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3d7d27-2ade-40b2-8e78-fb8fd606a324 None admin] RBAC: Adding query filter params (interface=public, service_id=c573224e5c174e158d887737842337cc, region_id=RegionOne) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:07:04.748551 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3d7d27-2ade-40b2-8e78-fb8fd606a324 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:04.750010 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3d7d27-2ade-40b2-8e78-fb8fd606a324 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:04.756204 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 530/1039] 10.222.0.23 () {56 vars in 1251 bytes} [Tue Jun 21 18:07:04 2022] GET /identity/v3/endpoints?service_id=c573224e5c174e158d887737842337cc&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:07:04.762612 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3d7d27-2ade-40b2-8e78-fb8fd606a324 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:04.810299 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59567c40-dadc-401c-9fed-6d2d0c64918b None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:04.835425 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:04.894773 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47b9447-a105-44a6-807d-dade4746fd39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:04.895940 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b9447-a105-44a6-807d-dade4746fd39 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:04.896448 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b9447-a105-44a6-807d-dade4746fd39 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:04.896947 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b9447-a105-44a6-807d-dade4746fd39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:04.903726 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3d7d27-2ade-40b2-8e78-fb8fd606a324 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:04.916945 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47b9447-a105-44a6-807d-dade4746fd39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:04.921051 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47b9447-a105-44a6-807d-dade4746fd39 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:04.947233 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-c47b9447-a105-44a6-807d-dade4746fd39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:07:04.947233 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:07:04.947233 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:07:04.947233 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:07:04.947233 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:07:04.947233 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:07:04.947233 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:07:04.947233 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:07:04.947233 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:07:04.947233 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:07:04.947233 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:07:04.947233 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:07:04.947233 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:07:04.947233 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:07:04.947233 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:07:04.948482 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:07:04.948482 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:07:04.948482 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:07:04.948482 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:07:04.948482 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:07:04.948482 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:07:04.948482 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:07:04.948482 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:07:04.948482 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:07:04.948482 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:07:04.948482 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:07:04.948482 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:07:04.948482 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:07:04.948482 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:07:04.948482 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:07:04.948482 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:07:04.948482 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:07:04.950022 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:07:04.950022 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:07:04.950022 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:07:04.950022 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:07:04.950022 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 510/1040] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:07:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:07:04.960203 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47b9447-a105-44a6-807d-dade4746fd39 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:04.963244 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d239913-bb68-4ee1-9af0-1561101cb4da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:04.964009 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d239913-bb68-4ee1-9af0-1561101cb4da None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:04.964299 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d239913-bb68-4ee1-9af0-1561101cb4da None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:04.964609 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d239913-bb68-4ee1-9af0-1561101cb4da None admin] PATH_INFO: `/v3/services` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:04.965351 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d239913-bb68-4ee1-9af0-1561101cb4da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:04.966759 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d239913-bb68-4ee1-9af0-1561101cb4da None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:04.976906 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 531/1041] 10.222.0.23 () {56 vars in 1087 bytes} [Tue Jun 21 18:07:04 2022] GET /identity/v3/services => generated 1957 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:07:05.028823 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:05.163859 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47b9447-a105-44a6-807d-dade4746fd39 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:05.206504 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00736e45-7c32-4800-8ab9-04308fd3ad14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:05.207183 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00736e45-7c32-4800-8ab9-04308fd3ad14 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:05.207462 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00736e45-7c32-4800-8ab9-04308fd3ad14 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:05.207728 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00736e45-7c32-4800-8ab9-04308fd3ad14 None admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:05.208495 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00736e45-7c32-4800-8ab9-04308fd3ad14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:07:05.208802 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00736e45-7c32-4800-8ab9-04308fd3ad14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:05.210197 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00736e45-7c32-4800-8ab9-04308fd3ad14 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:05.219293 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9331ca0-0c5c-4e38-b474-965f76329f14 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:05.219578 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9331ca0-0c5c-4e38-b474-965f76329f14 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:05.219910 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9331ca0-0c5c-4e38-b474-965f76329f14 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:05.224952 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 532/1042] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:07:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:07:05.225206 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 511/1043] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:07:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:07:05.230093 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9331ca0-0c5c-4e38-b474-965f76329f14 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:05.234172 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef0e80-9493-411b-a729-66d0949894dc None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:05.234465 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef0e80-9493-411b-a729-66d0949894dc None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:05.234751 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef0e80-9493-411b-a729-66d0949894dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:05.258100 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-38ef0e80-9493-411b-a729-66d0949894dc None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:07:05.260738 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-38ef0e80-9493-411b-a729-66d0949894dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:07:05.347106 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 512/1044] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:07:05 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:07:05.369064 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9331ca0-0c5c-4e38-b474-965f76329f14 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:05.372064 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ef0e80-9493-411b-a729-66d0949894dc None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:05.404029 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:05.419930 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:05.420123 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4d6fd4-3d1b-4943-b580-5a4c5443c1f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:05.420688 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4d6fd4-3d1b-4943-b580-5a4c5443c1f8 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:05.420975 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4d6fd4-3d1b-4943-b580-5a4c5443c1f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:05.421234 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4d6fd4-3d1b-4943-b580-5a4c5443c1f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:05.429184 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4d6fd4-3d1b-4943-b580-5a4c5443c1f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:05.431348 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4d6fd4-3d1b-4943-b580-5a4c5443c1f8 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:05.446799 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-9b4d6fd4-3d1b-4943-b580-5a4c5443c1f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:07:05.446799 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:07:05.446799 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:07:05.446799 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:07:05.446799 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:07:05.446799 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:07:05.446799 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:07:05.446799 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:07:05.446799 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:07:05.446799 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:07:05.446799 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:07:05.446799 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:07:05.446799 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:07:05.446799 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:07:05.446799 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:07:05.456769 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:07:05.456769 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:07:05.456769 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:07:05.456769 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:07:05.456769 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:07:05.456769 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:07:05.456769 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:07:05.456769 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:07:05.456769 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:07:05.456769 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:07:05.456769 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:07:05.456769 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:07:05.456769 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:07:05.456769 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:07:05.456769 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:07:05.456769 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:07:05.456769 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:07:05.457867 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:07:05.457867 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:07:05.457867 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:07:05.457867 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:07:05.457867 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:07:05.457867 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:07:05.457867 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:07:05.457867 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:07:05.457867 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 533/1045] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jun 21 18:07:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:07:05.457867 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4d6fd4-3d1b-4943-b580-5a4c5443c1f8 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:05.516151 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ef0e80-9493-411b-a729-66d0949894dc None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:05.556945 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef4701-bf9f-4235-b990-52fc9d290db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:05.557749 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef4701-bf9f-4235-b990-52fc9d290db6 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:05.558045 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef4701-bf9f-4235-b990-52fc9d290db6 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:05.558318 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef4701-bf9f-4235-b990-52fc9d290db6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:05.570380 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bef4701-bf9f-4235-b990-52fc9d290db6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:07:05.578322 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-6bef4701-bf9f-4235-b990-52fc9d290db6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:07:05.578322 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:07:05.578322 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:07:05.578322 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:07:05.578322 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:07:05.578322 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:07:05.578322 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:07:05.578322 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:07:05.578322 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:07:05.578322 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:07:05.578322 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:07:05.578322 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:07:05.578322 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:07:05.578322 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:07:05.578322 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:07:05.578322 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:07:05.579465 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:07:05.579465 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:07:05.579465 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:07:05.579465 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:07:05.579465 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:07:05.579465 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:07:05.579465 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:07:05.579465 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:07:05.579465 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:07:05.579465 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:07:05.579465 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:07:05.579465 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:07:05.579465 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:07:05.579465 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:07:05.579465 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:07:05.579465 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:07:05.581493 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:07:05.581493 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:07:05.581493 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:07:05.581493 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:07:05.581493 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:07:05.581493 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:07:05.581493 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:07:05.581493 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:07:05.581493 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:07:05.581493 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:07:05.581493 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:07:05.581493 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:07:05.581493 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 513/1046] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jun 21 18:07:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:07:05.585734 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef4701-bf9f-4235-b990-52fc9d290db6 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:05.603772 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4d6fd4-3d1b-4943-b580-5a4c5443c1f8 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:05.651810 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55530688-e4bc-4fe8-b055-85816def32ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:05.653094 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55530688-e4bc-4fe8-b055-85816def32ff None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:05.653865 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55530688-e4bc-4fe8-b055-85816def32ff None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:05.654411 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55530688-e4bc-4fe8-b055-85816def32ff None admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:05.655516 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55530688-e4bc-4fe8-b055-85816def32ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:07:05.656219 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55530688-e4bc-4fe8-b055-85816def32ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:05.658432 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55530688-e4bc-4fe8-b055-85816def32ff None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:05.675101 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 534/1047] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jun 21 18:07:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:07:05.682292 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55530688-e4bc-4fe8-b055-85816def32ff None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:05.698225 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:05.717201 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef4701-bf9f-4235-b990-52fc9d290db6 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:05.745619 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:05.762344 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3674b6-e96b-46dd-97ec-753bea57f5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:05.763234 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3674b6-e96b-46dd-97ec-753bea57f5a6 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:05.763536 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3674b6-e96b-46dd-97ec-753bea57f5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:05.763788 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3674b6-e96b-46dd-97ec-753bea57f5a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:05.764592 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3674b6-e96b-46dd-97ec-753bea57f5a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:07:05.764898 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3674b6-e96b-46dd-97ec-753bea57f5a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:05.766333 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3674b6-e96b-46dd-97ec-753bea57f5a6 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:05.776994 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 514/1048] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:07:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:07:05.782880 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3674b6-e96b-46dd-97ec-753bea57f5a6 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:05.828256 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55530688-e4bc-4fe8-b055-85816def32ff None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:05.880682 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b205bdbd-ad88-4e53-883e-9f5ce1f2a249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:05.881908 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205bdbd-ad88-4e53-883e-9f5ce1f2a249 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:05.882423 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205bdbd-ad88-4e53-883e-9f5ce1f2a249 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:05.882917 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205bdbd-ad88-4e53-883e-9f5ce1f2a249 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:05.884151 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b205bdbd-ad88-4e53-883e-9f5ce1f2a249 None admin] RBAC: Adding query filter params (role.id=e950e7aaae294af383fc9e83eed8250f, scope.project.id=2a4699d1ec70456da5f8e0aa4b1439b7, user.id=904255dd3c1f406eb10f6fc65f9cd098) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:07:05.885190 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b205bdbd-ad88-4e53-883e-9f5ce1f2a249 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:05.887977 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b205bdbd-ad88-4e53-883e-9f5ce1f2a249 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:05.897886 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 535/1049] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jun 21 18:07:05 2022] GET /identity/v3/role_assignments?user.id=904255dd3c1f406eb10f6fc65f9cd098&scope.project.id=2a4699d1ec70456da5f8e0aa4b1439b7&role.id=e950e7aaae294af383fc9e83eed8250f => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:07:05.950190 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3674b6-e96b-46dd-97ec-753bea57f5a6 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:06.008929 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df04cf20-7c38-47f4-b89e-c41cdc4cf4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:06.010379 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04cf20-7c38-47f4-b89e-c41cdc4cf4f5 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:06.010803 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04cf20-7c38-47f4-b89e-c41cdc4cf4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:06.011221 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04cf20-7c38-47f4-b89e-c41cdc4cf4f5 None admin] PATH_INFO: `/v3/users/b8d707ccbb034e6cbf7fedf6c5e40b4c` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:06.035851 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df04cf20-7c38-47f4-b89e-c41cdc4cf4f5 None admin] RBAC: Authorizing `identity:get_user(user_id=b8d707ccbb034e6cbf7fedf6c5e40b4c)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:06.038405 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df04cf20-7c38-47f4-b89e-c41cdc4cf4f5 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:06.047402 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 515/1050] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jun 21 18:07:05 2022] GET /identity/v3/users/b8d707ccbb034e6cbf7fedf6c5e40b4c => generated 285 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:07:06.048301 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b205bdbd-ad88-4e53-883e-9f5ce1f2a249 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:06.177891 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b205bdbd-ad88-4e53-883e-9f5ce1f2a249 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:06.183133 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:06.243379 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9fa2de-e034-445f-bfa6-e1288af99918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:06.244403 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9fa2de-e034-445f-bfa6-e1288af99918 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:06.244914 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9fa2de-e034-445f-bfa6-e1288af99918 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:06.245408 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9fa2de-e034-445f-bfa6-e1288af99918 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:06.374106 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9fa2de-e034-445f-bfa6-e1288af99918 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:06.376927 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9fa2de-e034-445f-bfa6-e1288af99918 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:06.506207 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 536/1051] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:07:06 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 18:07:06.513211 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df04cf20-7c38-47f4-b89e-c41cdc4cf4f5 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:06.530055 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:06.708661 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df04cf20-7c38-47f4-b89e-c41cdc4cf4f5 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:06.775046 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26812a35-d908-4539-ae72-ab88c072030a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:06.776772 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26812a35-d908-4539-ae72-ab88c072030a None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:06.777274 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26812a35-d908-4539-ae72-ab88c072030a None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:06.777766 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26812a35-d908-4539-ae72-ab88c072030a None admin] PATH_INFO: `/v3/projects/c7bd5615d37a425cb7bed3985d3eb711` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:06.780236 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26812a35-d908-4539-ae72-ab88c072030a None admin] RBAC: Authorizing `identity:get_project(project_id=c7bd5615d37a425cb7bed3985d3eb711)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:06.783139 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26812a35-d908-4539-ae72-ab88c072030a None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:06.792532 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 516/1052] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jun 21 18:07:06 2022] GET /identity/v3/projects/c7bd5615d37a425cb7bed3985d3eb711 => generated 311 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:07:06.797237 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9fa2de-e034-445f-bfa6-e1288af99918 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:06.804541 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:07.015309 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9fa2de-e034-445f-bfa6-e1288af99918 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:07.069083 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc744a5-01e4-459d-9b06-ec51118b572c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:07.069654 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc744a5-01e4-459d-9b06-ec51118b572c None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:07.069947 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc744a5-01e4-459d-9b06-ec51118b572c None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:07.070198 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc744a5-01e4-459d-9b06-ec51118b572c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:07.071746 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc744a5-01e4-459d-9b06-ec51118b572c None admin] RBAC: Adding query filter params (role.id=745ed9065a8143e384ebbd694b29635c, scope.project.id=c7bd5615d37a425cb7bed3985d3eb711, user.id=b8d707ccbb034e6cbf7fedf6c5e40b4c) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:07:07.072037 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc744a5-01e4-459d-9b06-ec51118b572c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:07.073457 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc744a5-01e4-459d-9b06-ec51118b572c None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:07.080201 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 537/1053] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jun 21 18:07:06 2022] GET /identity/v3/role_assignments?user.id=b8d707ccbb034e6cbf7fedf6c5e40b4c&scope.project.id=c7bd5615d37a425cb7bed3985d3eb711&role.id=745ed9065a8143e384ebbd694b29635c => generated 259 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:07:07.834053 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7372681e-fb93-421b-8eb7-ad066180423f None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:07.834910 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7372681e-fb93-421b-8eb7-ad066180423f None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:07.834910 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7372681e-fb93-421b-8eb7-ad066180423f None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:07.835986 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 517/1054] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:07:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:07:07.843225 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c45c519-31ae-486d-b773-abd4ce609a6d None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:07.843970 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c45c519-31ae-486d-b773-abd4ce609a6d None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:07.843970 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c45c519-31ae-486d-b773-abd4ce609a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:07.863983 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-4c45c519-31ae-486d-b773-abd4ce609a6d None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:07:07.865209 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-4c45c519-31ae-486d-b773-abd4ce609a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:07:07.948892 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:07.955770 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 538/1055] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:07:07 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:07:07.981813 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7372681e-fb93-421b-8eb7-ad066180423f None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:08.119463 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:08.167910 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7372681e-fb93-421b-8eb7-ad066180423f None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:08.243324 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd5aaf5-2d8c-4639-9058-493f05ca2fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:08.244785 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5aaf5-2d8c-4639-9058-493f05ca2fbd None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:08.245293 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5aaf5-2d8c-4639-9058-493f05ca2fbd None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:08.245789 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5aaf5-2d8c-4639-9058-493f05ca2fbd None admin] PATH_INFO: `/v3/services/image` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:08.247009 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd5aaf5-2d8c-4639-9058-493f05ca2fbd None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:08.249621 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd5aaf5-2d8c-4639-9058-493f05ca2fbd None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:08.272572 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-5bd5aaf5-2d8c-4639-9058-493f05ca2fbd None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 21 18:07:08.272572 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:07:08.272572 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:07:08.272572 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:07:08.272572 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:07:08.272572 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:07:08.272572 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:07:08.272572 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:07:08.272572 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 21 18:07:08.272572 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:07:08.272572 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:07:08.272572 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:07:08.272572 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:07:08.272572 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:07:08.272572 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:07:08.272572 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:07:08.272572 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:07:08.274592 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:07:08.274592 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:07:08.274592 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:07:08.274592 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:07:08.274592 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:07:08.274592 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:07:08.274592 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:07:08.274592 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:07:08.274592 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:07:08.274592 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:07:08.274592 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:07:08.274592 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:07:08.274592 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:07:08.274592 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:07:08.274592 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:07:08.274592 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:07:08.276289 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:07:08.276289 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:07:08.276289 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:07:08.276289 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:07:08.276289 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:07:08.276289 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:07:08.276289 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:07:08.276289 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:07:08.276289 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:07:08.276289 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:07:08.276289 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:07:08.276289 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:07:08.276289 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 21 18:07:08.276289 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:07:08.280334 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 518/1056] 10.222.0.23 () {56 vars in 1105 bytes} [Tue Jun 21 18:07:07 2022] GET /identity/v3/services/image => generated 86 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:07:08.285163 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c45c519-31ae-486d-b773-abd4ce609a6d None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:08.485857 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c45c519-31ae-486d-b773-abd4ce609a6d None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:08.552621 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0461e963-ce16-4a5e-8338-f84d99b8fc69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:08.553754 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0461e963-ce16-4a5e-8338-f84d99b8fc69 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:08.554258 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0461e963-ce16-4a5e-8338-f84d99b8fc69 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:08.554752 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0461e963-ce16-4a5e-8338-f84d99b8fc69 None admin] PATH_INFO: `/v3/services` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:08.556137 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0461e963-ce16-4a5e-8338-f84d99b8fc69 None admin] RBAC: Adding query filter params (name=image) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:07:08.556720 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0461e963-ce16-4a5e-8338-f84d99b8fc69 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:08.559276 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0461e963-ce16-4a5e-8338-f84d99b8fc69 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:08.576501 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 539/1057] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:07:08 2022] GET /identity/v3/services?name=image => generated 122 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:07:08.579535 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd5aaf5-2d8c-4639-9058-493f05ca2fbd None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:08.675780 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12497b9-0f47-4bed-ad00-8edfd9f89c7d None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:08.676277 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12497b9-0f47-4bed-ad00-8edfd9f89c7d None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:08.676819 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12497b9-0f47-4bed-ad00-8edfd9f89c7d None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:08.681104 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 540/1058] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:07:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:07:08.689647 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379d6528-10aa-4b54-a14e-104b68e29da1 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:08.690145 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379d6528-10aa-4b54-a14e-104b68e29da1 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:08.690633 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379d6528-10aa-4b54-a14e-104b68e29da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:08.718352 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-379d6528-10aa-4b54-a14e-104b68e29da1 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:07:08.720442 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-379d6528-10aa-4b54-a14e-104b68e29da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:07:08.745912 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd5aaf5-2d8c-4639-9058-493f05ca2fbd None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:08.807668 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25123890-8e0a-4436-8af5-f090e9699b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:08.808435 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25123890-8e0a-4436-8af5-f090e9699b7a None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:08.808783 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25123890-8e0a-4436-8af5-f090e9699b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:08.809105 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25123890-8e0a-4436-8af5-f090e9699b7a None admin] PATH_INFO: `/v3/services` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:08.809893 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25123890-8e0a-4436-8af5-f090e9699b7a None admin] RBAC: Adding query filter params (type=image) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:07:08.810233 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25123890-8e0a-4436-8af5-f090e9699b7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:08.811702 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25123890-8e0a-4436-8af5-f090e9699b7a None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:08.818296 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 519/1059] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:07:08 2022] GET /identity/v3/services?type=image => generated 350 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:07:08.824974 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25123890-8e0a-4436-8af5-f090e9699b7a None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:08.826486 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:08.828961 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:08.856488 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 541/1060] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:07:08 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:07:08.888376 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379d6528-10aa-4b54-a14e-104b68e29da1 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:08.995560 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25123890-8e0a-4436-8af5-f090e9699b7a None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:09.052966 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d451ff4-6ec2-4077-bc43-45c7a76963c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:09.053604 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d451ff4-6ec2-4077-bc43-45c7a76963c4 None admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:09.053914 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d451ff4-6ec2-4077-bc43-45c7a76963c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:09.054204 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d451ff4-6ec2-4077-bc43-45c7a76963c4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:09.054915 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d451ff4-6ec2-4077-bc43-45c7a76963c4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:09.057225 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d451ff4-6ec2-4077-bc43-45c7a76963c4 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:09.082604 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379d6528-10aa-4b54-a14e-104b68e29da1 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:09.140922 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cae64e-9920-46f9-b785-6756e203b5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:09.141834 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cae64e-9920-46f9-b785-6756e203b5b6 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:09.142181 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cae64e-9920-46f9-b785-6756e203b5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:09.142518 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cae64e-9920-46f9-b785-6756e203b5b6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:09.155871 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cae64e-9920-46f9-b785-6756e203b5b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:07:09.164416 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-73cae64e-9920-46f9-b785-6756e203b5b6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 18:07:09.164416 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:07:09.164416 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:07:09.164416 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:07:09.164416 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:07:09.164416 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:07:09.164416 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:07:09.164416 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:07:09.164416 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:07:09.164416 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:07:09.164416 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:07:09.164416 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:07:09.164416 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:07:09.164416 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:07:09.164416 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:07:09.164416 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:07:09.173317 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:07:09.173317 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:07:09.173317 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:07:09.173317 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:07:09.173317 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:07:09.173317 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:07:09.173317 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:07:09.173317 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:07:09.173317 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:07:09.173317 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:07:09.173317 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:07:09.173317 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:07:09.173317 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:07:09.173317 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:07:09.173317 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:07:09.173317 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:07:09.175046 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:07:09.175046 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:07:09.175046 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:07:09.175046 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:07:09.175046 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:07:09.175046 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:07:09.175046 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:07:09.175046 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:07:09.175046 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:07:09.175046 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:07:09.175046 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 18:07:09.175046 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:07:09.175046 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 542/1061] 10.222.0.23 () {56 vars in 1120 bytes} [Tue Jun 21 18:07:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:07:09.177583 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cae64e-9920-46f9-b785-6756e203b5b6 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:09.193278 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 520/1062] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Jun 21 18:07:08 2022] POST /identity/v3/endpoints => generated 327 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:07:09.193278 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c252d58c-94b7-4cba-84ae-e79333335e45 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:09.193278 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c252d58c-94b7-4cba-84ae-e79333335e45 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:09.193278 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c252d58c-94b7-4cba-84ae-e79333335e45 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:09.193278 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 521/1063] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:07:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:07:09.197403 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0890727c-1410-4fb3-a6c1-f885b8ebb185 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:09.197403 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0890727c-1410-4fb3-a6c1-f885b8ebb185 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:09.197403 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0890727c-1410-4fb3-a6c1-f885b8ebb185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:09.217575 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-0890727c-1410-4fb3-a6c1-f885b8ebb185 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:07:09.218842 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-0890727c-1410-4fb3-a6c1-f885b8ebb185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:07:09.283252 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:09.306572 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:09.342747 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 522/1064] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:07:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:07:09.368601 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0890727c-1410-4fb3-a6c1-f885b8ebb185 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:09.381354 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cae64e-9920-46f9-b785-6756e203b5b6 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:09.452005 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4b705b-c7be-4c6a-84c8-a205829ac899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:09.453138 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4b705b-c7be-4c6a-84c8-a205829ac899 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:09.453579 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4b705b-c7be-4c6a-84c8-a205829ac899 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:09.454035 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4b705b-c7be-4c6a-84c8-a205829ac899 None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:09.455320 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4b705b-c7be-4c6a-84c8-a205829ac899 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:07:09.455827 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4b705b-c7be-4c6a-84c8-a205829ac899 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:09.458118 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4b705b-c7be-4c6a-84c8-a205829ac899 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:09.471823 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 543/1065] 10.222.0.23 () {56 vars in 1115 bytes} [Tue Jun 21 18:07:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:07:09.479281 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4b705b-c7be-4c6a-84c8-a205829ac899 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:09.521163 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0890727c-1410-4fb3-a6c1-f885b8ebb185 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:09.571442 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cedbf5-3f3b-4462-a3f7-29e58dd52419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:09.571442 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cedbf5-3f3b-4462-a3f7-29e58dd52419 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:09.571442 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cedbf5-3f3b-4462-a3f7-29e58dd52419 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:09.571442 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cedbf5-3f3b-4462-a3f7-29e58dd52419 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:09.585663 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cedbf5-3f3b-4462-a3f7-29e58dd52419 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:07:09.595919 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-42cedbf5-3f3b-4462-a3f7-29e58dd52419 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:07:09.595919 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:07:09.595919 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:07:09.595919 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:07:09.595919 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:07:09.595919 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:07:09.595919 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:07:09.595919 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:07:09.595919 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:07:09.595919 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:07:09.595919 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:07:09.595919 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:07:09.595919 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:07:09.595919 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:07:09.595919 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:07:09.595919 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:07:09.598321 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:07:09.598321 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:07:09.598321 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:07:09.598321 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:07:09.598321 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:07:09.598321 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:07:09.598321 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:07:09.598321 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:07:09.598321 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:07:09.598321 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:07:09.598321 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:07:09.598321 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:07:09.598321 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:07:09.598321 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:07:09.598321 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:07:09.598321 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:07:09.600920 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:07:09.600920 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:07:09.600920 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:07:09.600920 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:07:09.600920 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:07:09.600920 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:07:09.600920 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:07:09.600920 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:07:09.600920 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:07:09.600920 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:07:09.600920 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:07:09.600920 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:07:09.600920 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 523/1066] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jun 21 18:07:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:07:09.607867 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cedbf5-3f3b-4462-a3f7-29e58dd52419 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:09.670013 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4b705b-c7be-4c6a-84c8-a205829ac899 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:09.670225 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:09.698376 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:09.722655 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c7f271-f420-414d-a995-d0360e6461a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:09.723676 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c7f271-f420-414d-a995-d0360e6461a1 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:09.723865 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c7f271-f420-414d-a995-d0360e6461a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:09.724178 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c7f271-f420-414d-a995-d0360e6461a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:09.743771 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c7f271-f420-414d-a995-d0360e6461a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:09.747026 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c7f271-f420-414d-a995-d0360e6461a1 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:09.757085 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cedbf5-3f3b-4462-a3f7-29e58dd52419 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:09.776041 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-b6c7f271-f420-414d-a995-d0360e6461a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:07:09.776041 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:07:09.776041 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:07:09.776041 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:07:09.776041 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:07:09.776041 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:07:09.776041 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:07:09.776041 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:07:09.776041 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:07:09.776041 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:07:09.776041 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:07:09.776041 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:07:09.776041 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:07:09.776041 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:07:09.776041 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:07:09.777174 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:07:09.777174 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:07:09.777174 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:07:09.777174 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:07:09.777174 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:07:09.777174 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:07:09.777174 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:07:09.777174 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:07:09.777174 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:07:09.777174 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:07:09.777174 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:07:09.777174 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:07:09.777174 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:07:09.777174 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:07:09.777174 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:07:09.777174 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:07:09.777174 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:07:09.778189 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:07:09.778189 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:07:09.778189 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:07:09.778189 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:07:09.778189 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 544/1067] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:07:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:07:09.783664 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c7f271-f420-414d-a995-d0360e6461a1 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:09.818098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca36e0c9-5f5b-453e-8a38-c1fbb5a20b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:09.819243 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca36e0c9-5f5b-453e-8a38-c1fbb5a20b6f None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:09.819741 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca36e0c9-5f5b-453e-8a38-c1fbb5a20b6f None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:09.820241 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca36e0c9-5f5b-453e-8a38-c1fbb5a20b6f None admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:09.822339 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca36e0c9-5f5b-453e-8a38-c1fbb5a20b6f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:07:09.822815 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca36e0c9-5f5b-453e-8a38-c1fbb5a20b6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:09.825329 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca36e0c9-5f5b-453e-8a38-c1fbb5a20b6f None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:09.849029 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 524/1068] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:07:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:07:09.850522 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca36e0c9-5f5b-453e-8a38-c1fbb5a20b6f None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:09.957632 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c7f271-f420-414d-a995-d0360e6461a1 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:10.002132 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f52785-24c4-488e-b0d2-dc19f4f4c296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:10.002817 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f52785-24c4-488e-b0d2-dc19f4f4c296 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:10.003100 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f52785-24c4-488e-b0d2-dc19f4f4c296 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:10.003365 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f52785-24c4-488e-b0d2-dc19f4f4c296 None admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:10.004101 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f52785-24c4-488e-b0d2-dc19f4f4c296 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:07:10.004421 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f52785-24c4-488e-b0d2-dc19f4f4c296 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:10.005858 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f52785-24c4-488e-b0d2-dc19f4f4c296 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:10.009763 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca36e0c9-5f5b-453e-8a38-c1fbb5a20b6f None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:10.021636 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 545/1069] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:07:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:07:10.025991 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:10.027906 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f52785-24c4-488e-b0d2-dc19f4f4c296 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:10.069543 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a82a522-2961-4b3e-bb6d-0be2c0e579bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:10.071178 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:10.072327 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a82a522-2961-4b3e-bb6d-0be2c0e579bc None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:10.073045 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a82a522-2961-4b3e-bb6d-0be2c0e579bc None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:10.073769 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a82a522-2961-4b3e-bb6d-0be2c0e579bc None admin] PATH_INFO: `/v3/users/b8d707ccbb034e6cbf7fedf6c5e40b4c` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:10.075896 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a82a522-2961-4b3e-bb6d-0be2c0e579bc None admin] RBAC: Authorizing `identity:get_user(user_id=b8d707ccbb034e6cbf7fedf6c5e40b4c)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:10.078812 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a82a522-2961-4b3e-bb6d-0be2c0e579bc None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:10.088939 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 525/1070] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jun 21 18:07:09 2022] GET /identity/v3/users/b8d707ccbb034e6cbf7fedf6c5e40b4c => generated 285 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:07:10.092424 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a82a522-2961-4b3e-bb6d-0be2c0e579bc None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:10.208878 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f52785-24c4-488e-b0d2-dc19f4f4c296 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:10.271698 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d294c59b-fa4d-4e82-bded-dce4cfb6b110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:10.273393 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d294c59b-fa4d-4e82-bded-dce4cfb6b110 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:10.274016 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d294c59b-fa4d-4e82-bded-dce4cfb6b110 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:10.274533 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d294c59b-fa4d-4e82-bded-dce4cfb6b110 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:10.283126 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a82a522-2961-4b3e-bb6d-0be2c0e579bc None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:10.288814 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d294c59b-fa4d-4e82-bded-dce4cfb6b110 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:10.292513 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d294c59b-fa4d-4e82-bded-dce4cfb6b110 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:10.325298 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-d294c59b-fa4d-4e82-bded-dce4cfb6b110 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 21 18:07:10.325298 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:07:10.325298 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:07:10.325298 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:07:10.325298 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:07:10.325298 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:07:10.325298 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:07:10.325298 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:07:10.325298 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:07:10.325298 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:07:10.325298 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:07:10.325298 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:07:10.325298 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:07:10.325298 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:07:10.325298 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:07:10.325298 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:07:10.326341 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:07:10.326341 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:07:10.326341 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:07:10.326341 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:07:10.326341 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:07:10.326341 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:07:10.326341 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:07:10.326341 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:07:10.326341 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:07:10.326341 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:07:10.326341 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:07:10.326341 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:07:10.326341 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:07:10.326341 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:07:10.326341 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:07:10.326341 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:07:10.326341 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:07:10.326341 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:07:10.327330 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:07:10.327330 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:07:10.327330 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:07:10.327330 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:07:10.327330 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:07:10.327330 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:07:10.327330 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:07:10.327330 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:07:10.327330 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 21 18:07:10.327330 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:07:10.327907 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 546/1071] 10.222.0.23 () {56 vars in 1093 bytes} [Tue Jun 21 18:07:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:07:10.337957 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d294c59b-fa4d-4e82-bded-dce4cfb6b110 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:10.352917 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355a5fc1-e291-42ae-85cd-c77a38fd0e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:10.354429 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355a5fc1-e291-42ae-85cd-c77a38fd0e68 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:10.354943 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355a5fc1-e291-42ae-85cd-c77a38fd0e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:10.355420 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355a5fc1-e291-42ae-85cd-c77a38fd0e68 None admin] PATH_INFO: `/v3/projects/c7bd5615d37a425cb7bed3985d3eb711` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:10.357764 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355a5fc1-e291-42ae-85cd-c77a38fd0e68 None admin] RBAC: Authorizing `identity:get_project(project_id=c7bd5615d37a425cb7bed3985d3eb711)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:10.360796 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355a5fc1-e291-42ae-85cd-c77a38fd0e68 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:10.363812 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 526/1072] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jun 21 18:07:10 2022] GET /identity/v3/projects/c7bd5615d37a425cb7bed3985d3eb711 => generated 311 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:07:10.370353 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355a5fc1-e291-42ae-85cd-c77a38fd0e68 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:10.431804 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:10.506002 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:10.528119 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d294c59b-fa4d-4e82-bded-dce4cfb6b110 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:10.535614 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355a5fc1-e291-42ae-85cd-c77a38fd0e68 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:10.582618 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4744115-51f5-41ae-a3c3-5ddd01df7ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:10.583687 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4744115-51f5-41ae-a3c3-5ddd01df7ded None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:10.584199 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4744115-51f5-41ae-a3c3-5ddd01df7ded None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:10.584727 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4744115-51f5-41ae-a3c3-5ddd01df7ded None admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:10.586944 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4744115-51f5-41ae-a3c3-5ddd01df7ded None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:07:10.587746 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4744115-51f5-41ae-a3c3-5ddd01df7ded None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:10.590614 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4744115-51f5-41ae-a3c3-5ddd01df7ded None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:10.604349 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e227c58b-6ed0-4475-a3cc-2c5a49b5b581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:10.605317 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e227c58b-6ed0-4475-a3cc-2c5a49b5b581 None admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:10.605593 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e227c58b-6ed0-4475-a3cc-2c5a49b5b581 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:10.605884 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e227c58b-6ed0-4475-a3cc-2c5a49b5b581 None admin] PATH_INFO: `/v3/projects/c7bd5615d37a425cb7bed3985d3eb711/users/b8d707ccbb034e6cbf7fedf6c5e40b4c/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:10.608325 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e227c58b-6ed0-4475-a3cc-2c5a49b5b581 None admin] RBAC: Authorizing `identity:create_grant(project_id=c7bd5615d37a425cb7bed3985d3eb711, user_id=b8d707ccbb034e6cbf7fedf6c5e40b4c, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:10.612283 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e227c58b-6ed0-4475-a3cc-2c5a49b5b581 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:10.632075 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 547/1073] 10.222.0.23 () {56 vars in 1133 bytes} [Tue Jun 21 18:07:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:07:10.634120 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4744115-51f5-41ae-a3c3-5ddd01df7ded None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:10.696086 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 527/1074] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jun 21 18:07:10 2022] PUT /identity/v3/projects/c7bd5615d37a425cb7bed3985d3eb711/users/b8d707ccbb034e6cbf7fedf6c5e40b4c/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:07:10.833051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4744115-51f5-41ae-a3c3-5ddd01df7ded None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:10.892504 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2a0840-a234-4d29-ae18-c8689fb1a54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:10.893325 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a0840-a234-4d29-ae18-c8689fb1a54b None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:10.893609 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a0840-a234-4d29-ae18-c8689fb1a54b None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:10.893883 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a0840-a234-4d29-ae18-c8689fb1a54b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:10.907394 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2a0840-a234-4d29-ae18-c8689fb1a54b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:10.908961 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2a0840-a234-4d29-ae18-c8689fb1a54b None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:10.912258 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:10.931709 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-6a2a0840-a234-4d29-ae18-c8689fb1a54b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:07:10.931709 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:07:10.931709 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:07:10.931709 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:07:10.931709 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:07:10.931709 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:07:10.931709 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:07:10.931709 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:07:10.931709 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:07:10.931709 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:07:10.931709 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:07:10.931709 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:07:10.931709 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:07:10.931709 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:07:10.931709 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:07:10.937383 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:07:10.937383 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:07:10.937383 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:07:10.937383 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:07:10.937383 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:07:10.937383 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:07:10.937383 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:07:10.937383 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:07:10.937383 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:07:10.937383 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:07:10.937383 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:07:10.937383 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:07:10.937383 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:07:10.937383 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:07:10.937383 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:07:10.937383 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:07:10.937383 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:07:10.939590 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:07:10.939590 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:07:10.939590 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:07:10.939590 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:07:10.939590 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 548/1075] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:07:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:07:10.949366 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e227c58b-6ed0-4475-a3cc-2c5a49b5b581 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:11.142695 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e227c58b-6ed0-4475-a3cc-2c5a49b5b581 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:11.145529 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:11.184858 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f36b31-18bf-426b-9d2b-f46f628760cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:11.185589 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f36b31-18bf-426b-9d2b-f46f628760cf None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:11.185877 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f36b31-18bf-426b-9d2b-f46f628760cf None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:11.186145 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f36b31-18bf-426b-9d2b-f46f628760cf None admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:11.186901 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f36b31-18bf-426b-9d2b-f46f628760cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:07:11.187219 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f36b31-18bf-426b-9d2b-f46f628760cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:11.188671 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f36b31-18bf-426b-9d2b-f46f628760cf None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:11.203744 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 528/1076] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:07:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:07:11.209417 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2a0840-a234-4d29-ae18-c8689fb1a54b None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:11.329862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2a0840-a234-4d29-ae18-c8689fb1a54b None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:11.373643 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aff6d96-a6c7-4f01-8b52-8f1669504dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:11.375133 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff6d96-a6c7-4f01-8b52-8f1669504dbd None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:11.375551 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff6d96-a6c7-4f01-8b52-8f1669504dbd None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:11.375956 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff6d96-a6c7-4f01-8b52-8f1669504dbd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:11.388869 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aff6d96-a6c7-4f01-8b52-8f1669504dbd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:11.391044 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aff6d96-a6c7-4f01-8b52-8f1669504dbd None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:11.413233 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-0aff6d96-a6c7-4f01-8b52-8f1669504dbd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:07:11.413233 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:07:11.413233 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:07:11.413233 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:07:11.413233 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:07:11.413233 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:07:11.413233 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:07:11.413233 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:07:11.413233 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:07:11.413233 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:07:11.413233 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:07:11.413233 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:07:11.413233 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:07:11.413233 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:07:11.413233 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:07:11.415111 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:07:11.415111 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:07:11.415111 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:07:11.415111 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:07:11.415111 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:07:11.415111 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:07:11.415111 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:07:11.415111 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:07:11.415111 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:07:11.415111 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:07:11.415111 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:07:11.415111 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:07:11.415111 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:07:11.415111 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:07:11.415111 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:07:11.415111 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:07:11.415111 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:07:11.418181 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:07:11.418181 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:07:11.418181 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:07:11.418181 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:07:11.418181 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:07:11.418181 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:07:11.418181 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:07:11.418181 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:07:11.420983 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 549/1077] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jun 21 18:07:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:07:11.423617 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f36b31-18bf-426b-9d2b-f46f628760cf None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:11.581454 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f36b31-18bf-426b-9d2b-f46f628760cf None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:11.629645 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffd359b-563d-4ecb-a45c-fec57b08795e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:11.630346 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffd359b-563d-4ecb-a45c-fec57b08795e None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:11.630632 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffd359b-563d-4ecb-a45c-fec57b08795e None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:11.630917 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffd359b-563d-4ecb-a45c-fec57b08795e None admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:11.631759 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffd359b-563d-4ecb-a45c-fec57b08795e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:07:11.632085 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffd359b-563d-4ecb-a45c-fec57b08795e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:11.634218 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffd359b-563d-4ecb-a45c-fec57b08795e None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:11.652117 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a2e24-9d97-451d-bec7-bd6560881987 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:11.652632 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a2e24-9d97-451d-bec7-bd6560881987 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:11.653066 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a2e24-9d97-451d-bec7-bd6560881987 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:11.655769 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 550/1078] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:07:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:07:11.659849 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 529/1079] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jun 21 18:07:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:07:11.665605 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab879781-f98c-4278-abf5-b9410e3c6144 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:11.665977 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab879781-f98c-4278-abf5-b9410e3c6144 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:11.666425 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab879781-f98c-4278-abf5-b9410e3c6144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:11.667840 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffd359b-563d-4ecb-a45c-fec57b08795e None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:11.697195 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-ab879781-f98c-4278-abf5-b9410e3c6144 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:07:11.699453 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-ab879781-f98c-4278-abf5-b9410e3c6144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:07:11.707675 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:11.793290 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:11.837768 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 551/1080] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:07:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:07:11.847060 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffd359b-563d-4ecb-a45c-fec57b08795e None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:11.874651 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab879781-f98c-4278-abf5-b9410e3c6144 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:11.928435 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49acc353-9cc2-435e-9926-77d4e76b8f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:11.929928 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49acc353-9cc2-435e-9926-77d4e76b8f5a None admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:11.931820 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49acc353-9cc2-435e-9926-77d4e76b8f5a None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:11.932504 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49acc353-9cc2-435e-9926-77d4e76b8f5a None admin] PATH_INFO: `/v3/projects/2a4699d1ec70456da5f8e0aa4b1439b7/users/904255dd3c1f406eb10f6fc65f9cd098/roles/e950e7aaae294af383fc9e83eed8250f` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:11.936537 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49acc353-9cc2-435e-9926-77d4e76b8f5a None admin] RBAC: Authorizing `identity:create_grant(project_id=2a4699d1ec70456da5f8e0aa4b1439b7, user_id=904255dd3c1f406eb10f6fc65f9cd098, role_id=e950e7aaae294af383fc9e83eed8250f)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:11.939296 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49acc353-9cc2-435e-9926-77d4e76b8f5a None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:12.026903 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab879781-f98c-4278-abf5-b9410e3c6144 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:12.034995 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 530/1081] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jun 21 18:07:11 2022] PUT /identity/v3/projects/2a4699d1ec70456da5f8e0aa4b1439b7/users/904255dd3c1f406eb10f6fc65f9cd098/roles/e950e7aaae294af383fc9e83eed8250f => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:07:12.073642 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39454305-129a-46af-bb51-296af336b158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:12.074334 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39454305-129a-46af-bb51-296af336b158 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:12.074616 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39454305-129a-46af-bb51-296af336b158 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:12.075028 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39454305-129a-46af-bb51-296af336b158 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:12.154927 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:12.174714 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39454305-129a-46af-bb51-296af336b158 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:12.176692 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39454305-129a-46af-bb51-296af336b158 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:12.330201 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 552/1082] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:07:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 21 18:07:12.340055 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49acc353-9cc2-435e-9926-77d4e76b8f5a None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:12.475385 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:12.509311 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49acc353-9cc2-435e-9926-77d4e76b8f5a None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:12.575057 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80202121-055a-428e-b58d-1a4c66d23d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:12.576506 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80202121-055a-428e-b58d-1a4c66d23d8d None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:12.577097 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80202121-055a-428e-b58d-1a4c66d23d8d None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:12.577615 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80202121-055a-428e-b58d-1a4c66d23d8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:12.595767 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80202121-055a-428e-b58d-1a4c66d23d8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:12.597161 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80202121-055a-428e-b58d-1a4c66d23d8d None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:12.619041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-80202121-055a-428e-b58d-1a4c66d23d8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:07:12.619041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:07:12.619041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:07:12.619041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:07:12.619041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:07:12.619041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:07:12.619041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:07:12.619041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:07:12.619041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:07:12.619041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:07:12.619041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:07:12.619041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:07:12.619041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:07:12.619041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:07:12.619041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:07:12.629732 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:07:12.629732 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:07:12.629732 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:07:12.629732 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:07:12.629732 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:07:12.629732 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:07:12.629732 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:07:12.629732 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:07:12.629732 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:07:12.629732 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:07:12.629732 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:07:12.629732 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:07:12.629732 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:07:12.629732 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:07:12.629732 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:07:12.629732 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:07:12.629732 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:07:12.631844 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:07:12.631844 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:07:12.631844 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:07:12.631844 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:07:12.631844 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 531/1083] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:07:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:07:12.634862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39454305-129a-46af-bb51-296af336b158 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:12.739274 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:12.758154 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39454305-129a-46af-bb51-296af336b158 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:12.799344 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a88241e-b17a-43cd-b4e4-e1beabf482c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:12.800228 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a88241e-b17a-43cd-b4e4-e1beabf482c5 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:12.800693 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a88241e-b17a-43cd-b4e4-e1beabf482c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:12.801104 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a88241e-b17a-43cd-b4e4-e1beabf482c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:12.801978 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a88241e-b17a-43cd-b4e4-e1beabf482c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:07:12.802448 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a88241e-b17a-43cd-b4e4-e1beabf482c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:12.803984 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a88241e-b17a-43cd-b4e4-e1beabf482c5 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:12.817796 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 553/1084] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:07:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:07:12.904533 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff21bad0-d95b-4c9e-931d-6b7c1ffe951b None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:12.904834 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff21bad0-d95b-4c9e-931d-6b7c1ffe951b None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:12.905199 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff21bad0-d95b-4c9e-931d-6b7c1ffe951b None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:12.906836 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 532/1085] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:07:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:07:12.912721 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc357a4f-2ed6-4bd6-b4eb-d729f1f85f4d None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:12.913107 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc357a4f-2ed6-4bd6-b4eb-d729f1f85f4d None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:12.913305 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc357a4f-2ed6-4bd6-b4eb-d729f1f85f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:12.934440 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-bc357a4f-2ed6-4bd6-b4eb-d729f1f85f4d None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:07:12.935827 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-bc357a4f-2ed6-4bd6-b4eb-d729f1f85f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:07:13.022985 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 554/1086] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:07:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:07:13.049056 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff21bad0-d95b-4c9e-931d-6b7c1ffe951b None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:13.258846 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff21bad0-d95b-4c9e-931d-6b7c1ffe951b None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:13.321300 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef62d9e9-7bff-4cbe-a837-76a3331e447a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:13.322928 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef62d9e9-7bff-4cbe-a837-76a3331e447a None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:13.323433 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef62d9e9-7bff-4cbe-a837-76a3331e447a None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:13.323924 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef62d9e9-7bff-4cbe-a837-76a3331e447a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:13.327406 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:13.340804 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef62d9e9-7bff-4cbe-a837-76a3331e447a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:07:13.349185 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-ef62d9e9-7bff-4cbe-a837-76a3331e447a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:07:13.349185 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:07:13.349185 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:07:13.349185 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:07:13.349185 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:07:13.349185 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:07:13.349185 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:07:13.349185 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:07:13.349185 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:07:13.349185 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:07:13.349185 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:07:13.349185 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:07:13.349185 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:07:13.349185 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:07:13.349185 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:07:13.349185 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:07:13.351260 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:07:13.351260 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:07:13.351260 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:07:13.351260 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:07:13.351260 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:07:13.351260 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:07:13.351260 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:07:13.351260 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:07:13.351260 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:07:13.351260 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:07:13.351260 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:07:13.351260 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:07:13.351260 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:07:13.351260 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:07:13.351260 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:07:13.351260 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:07:13.352618 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:07:13.352618 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:07:13.352618 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:07:13.352618 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:07:13.352618 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:07:13.352618 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:07:13.352618 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:07:13.352618 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:07:13.352618 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:07:13.352618 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:07:13.352618 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:07:13.352618 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:07:13.352618 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 533/1087] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jun 21 18:07:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:07:13.361800 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc357a4f-2ed6-4bd6-b4eb-d729f1f85f4d None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:13.455965 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:13.484057 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc357a4f-2ed6-4bd6-b4eb-d729f1f85f4d None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:13.525783 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f73a477-847b-42e3-95f0-19e6d72a419d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:13.526538 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f73a477-847b-42e3-95f0-19e6d72a419d None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:13.526917 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f73a477-847b-42e3-95f0-19e6d72a419d None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:13.527291 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f73a477-847b-42e3-95f0-19e6d72a419d None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:13.528274 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f73a477-847b-42e3-95f0-19e6d72a419d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:07:13.528735 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f73a477-847b-42e3-95f0-19e6d72a419d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:13.530352 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f73a477-847b-42e3-95f0-19e6d72a419d None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:13.541044 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 555/1088] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:07:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:07:13.547127 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef62d9e9-7bff-4cbe-a837-76a3331e447a None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:13.672928 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef62d9e9-7bff-4cbe-a837-76a3331e447a None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:13.717253 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16a57b4-2985-44b1-b6e7-c561f24d29ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:13.718178 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a57b4-2985-44b1-b6e7-c561f24d29ed None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:13.718464 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a57b4-2985-44b1-b6e7-c561f24d29ed None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:13.718746 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a57b4-2985-44b1-b6e7-c561f24d29ed None admin] PATH_INFO: `/v3/users/b8d707ccbb034e6cbf7fedf6c5e40b4c` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:13.720175 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16a57b4-2985-44b1-b6e7-c561f24d29ed None admin] RBAC: Authorizing `identity:get_user(user_id=b8d707ccbb034e6cbf7fedf6c5e40b4c)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:13.721712 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16a57b4-2985-44b1-b6e7-c561f24d29ed None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:13.723486 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 534/1089] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jun 21 18:07:13 2022] GET /identity/v3/users/b8d707ccbb034e6cbf7fedf6c5e40b4c => generated 285 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:07:13.733293 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f73a477-847b-42e3-95f0-19e6d72a419d None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:13.856614 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f73a477-847b-42e3-95f0-19e6d72a419d None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:13.918291 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fc9277-4ae7-4ff7-8e57-bc316e6672ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:13.919341 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc9277-4ae7-4ff7-8e57-bc316e6672ca None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:13.919810 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc9277-4ae7-4ff7-8e57-bc316e6672ca None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:13.920302 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc9277-4ae7-4ff7-8e57-bc316e6672ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:13.938699 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:14.023621 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fc9277-4ae7-4ff7-8e57-bc316e6672ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:14.025708 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fc9277-4ae7-4ff7-8e57-bc316e6672ca None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:14.214452 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 556/1090] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:07:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 21 18:07:14.216379 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16a57b4-2985-44b1-b6e7-c561f24d29ed None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:14.331998 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:14.385094 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16a57b4-2985-44b1-b6e7-c561f24d29ed None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:14.424924 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49636807-7f32-4f0a-bf06-822aeb228495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:14.425790 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49636807-7f32-4f0a-bf06-822aeb228495 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:14.426065 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49636807-7f32-4f0a-bf06-822aeb228495 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:14.426334 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49636807-7f32-4f0a-bf06-822aeb228495 None admin] PATH_INFO: `/v3/projects/c7bd5615d37a425cb7bed3985d3eb711` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:14.427592 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49636807-7f32-4f0a-bf06-822aeb228495 None admin] RBAC: Authorizing `identity:get_project(project_id=c7bd5615d37a425cb7bed3985d3eb711)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:14.429088 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49636807-7f32-4f0a-bf06-822aeb228495 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:14.432569 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 535/1091] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jun 21 18:07:14 2022] GET /identity/v3/projects/c7bd5615d37a425cb7bed3985d3eb711 => generated 311 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:07:14.439337 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fc9277-4ae7-4ff7-8e57-bc316e6672ca None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:14.498471 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:14.505463 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d60a8-b8b8-42df-a511-8bac65e9bf86 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:14.505886 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d60a8-b8b8-42df-a511-8bac65e9bf86 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:14.506405 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d60a8-b8b8-42df-a511-8bac65e9bf86 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:14.507910 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 536/1092] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:07:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:07:14.514338 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae4658-7a22-4928-8508-33555c2a6e0f None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:14.514628 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae4658-7a22-4928-8508-33555c2a6e0f None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:14.514910 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae4658-7a22-4928-8508-33555c2a6e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:14.531846 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-f6ae4658-7a22-4928-8508-33555c2a6e0f None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:07:14.533107 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-f6ae4658-7a22-4928-8508-33555c2a6e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:07:14.567097 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fc9277-4ae7-4ff7-8e57-bc316e6672ca None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:14.603900 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b60c030-625e-4eca-9279-0e62a8fbd702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:14.604598 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b60c030-625e-4eca-9279-0e62a8fbd702 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:14.604881 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b60c030-625e-4eca-9279-0e62a8fbd702 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:14.605155 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b60c030-625e-4eca-9279-0e62a8fbd702 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:14.606007 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b60c030-625e-4eca-9279-0e62a8fbd702 None admin] RBAC: Adding query filter params (role.id=745ed9065a8143e384ebbd694b29635c, scope.project.id=c7bd5615d37a425cb7bed3985d3eb711, user.id=b8d707ccbb034e6cbf7fedf6c5e40b4c) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:07:14.606320 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b60c030-625e-4eca-9279-0e62a8fbd702 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:14.607795 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b60c030-625e-4eca-9279-0e62a8fbd702 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:14.610149 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 537/1093] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:07:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:07:14.614909 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 557/1094] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jun 21 18:07:14 2022] GET /identity/v3/role_assignments?user.id=b8d707ccbb034e6cbf7fedf6c5e40b4c&scope.project.id=c7bd5615d37a425cb7bed3985d3eb711&role.id=745ed9065a8143e384ebbd694b29635c => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:07:14.636868 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae4658-7a22-4928-8508-33555c2a6e0f None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:14.732224 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:14.790742 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae4658-7a22-4928-8508-33555c2a6e0f None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:14.835237 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c260a90-f20c-4ee1-b958-3c9154962f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:14.836906 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c260a90-f20c-4ee1-b958-3c9154962f04 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:14.837906 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c260a90-f20c-4ee1-b958-3c9154962f04 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:14.838321 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c260a90-f20c-4ee1-b958-3c9154962f04 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:14.856190 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c260a90-f20c-4ee1-b958-3c9154962f04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:07:14.869637 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-8c260a90-f20c-4ee1-b958-3c9154962f04 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 18:07:14.869637 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:07:14.869637 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:07:14.869637 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:07:14.869637 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:07:14.869637 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:07:14.869637 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:07:14.869637 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:07:14.869637 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:07:14.869637 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:07:14.869637 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:07:14.869637 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:07:14.869637 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:07:14.869637 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:07:14.869637 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:07:14.869637 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:07:14.871750 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:07:14.871750 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:07:14.871750 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:07:14.871750 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:07:14.871750 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:07:14.871750 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:07:14.871750 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:07:14.871750 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:07:14.871750 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:07:14.871750 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:07:14.871750 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:07:14.871750 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:07:14.871750 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:07:14.871750 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:07:14.871750 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:07:14.871750 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:07:14.873054 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:07:14.873054 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:07:14.873054 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:07:14.873054 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:07:14.873054 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:07:14.873054 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:07:14.873054 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:07:14.873054 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:07:14.873054 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:07:14.873054 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:07:14.873054 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 18:07:14.873054 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:07:14.873054 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 538/1095] 10.222.0.23 () {56 vars in 1120 bytes} [Tue Jun 21 18:07:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:07:14.879742 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b60c030-625e-4eca-9279-0e62a8fbd702 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:14.999572 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b60c030-625e-4eca-9279-0e62a8fbd702 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:15.001474 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:15.036798 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19400a7f-8da4-4231-9b55-2d3de54d081d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:15.037486 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19400a7f-8da4-4231-9b55-2d3de54d081d None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:15.037766 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19400a7f-8da4-4231-9b55-2d3de54d081d None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:15.038037 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19400a7f-8da4-4231-9b55-2d3de54d081d None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:15.038794 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19400a7f-8da4-4231-9b55-2d3de54d081d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:07:15.039096 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19400a7f-8da4-4231-9b55-2d3de54d081d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:15.041106 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19400a7f-8da4-4231-9b55-2d3de54d081d None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:15.049983 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 558/1096] 10.222.0.23 () {56 vars in 1115 bytes} [Tue Jun 21 18:07:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:07:15.057200 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c260a90-f20c-4ee1-b958-3c9154962f04 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:15.170529 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c260a90-f20c-4ee1-b958-3c9154962f04 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:15.194390 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:15.210068 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027cf1cd-06e9-4aee-9070-051fc70f20b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:15.210877 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027cf1cd-06e9-4aee-9070-051fc70f20b7 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:15.211159 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027cf1cd-06e9-4aee-9070-051fc70f20b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:15.211473 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027cf1cd-06e9-4aee-9070-051fc70f20b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:15.223144 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027cf1cd-06e9-4aee-9070-051fc70f20b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:15.224888 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027cf1cd-06e9-4aee-9070-051fc70f20b7 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:15.244301 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-027cf1cd-06e9-4aee-9070-051fc70f20b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:07:15.244301 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:07:15.244301 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:07:15.244301 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:07:15.244301 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:07:15.244301 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:07:15.244301 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:07:15.244301 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:07:15.244301 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:07:15.244301 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:07:15.244301 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:07:15.244301 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:07:15.244301 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:07:15.244301 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:07:15.244301 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:07:15.246171 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:07:15.246171 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:07:15.246171 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:07:15.246171 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:07:15.246171 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:07:15.246171 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:07:15.246171 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:07:15.246171 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:07:15.246171 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:07:15.246171 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:07:15.246171 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:07:15.246171 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:07:15.246171 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:07:15.246171 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:07:15.246171 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:07:15.246171 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:07:15.246171 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:07:15.247943 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:07:15.247943 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:07:15.247943 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:07:15.247943 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:07:15.247943 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 539/1097] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:07:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:07:15.252728 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19400a7f-8da4-4231-9b55-2d3de54d081d None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:15.365620 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19400a7f-8da4-4231-9b55-2d3de54d081d None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:15.403188 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c96adb-54d5-4ace-9a99-15ba29a4928b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:15.403878 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c96adb-54d5-4ace-9a99-15ba29a4928b None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:15.404158 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c96adb-54d5-4ace-9a99-15ba29a4928b None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:15.404464 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c96adb-54d5-4ace-9a99-15ba29a4928b None admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:15.405204 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c96adb-54d5-4ace-9a99-15ba29a4928b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:07:15.405510 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c96adb-54d5-4ace-9a99-15ba29a4928b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:15.406925 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c96adb-54d5-4ace-9a99-15ba29a4928b None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:15.421277 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 559/1098] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:07:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:07:15.427099 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027cf1cd-06e9-4aee-9070-051fc70f20b7 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:15.541052 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027cf1cd-06e9-4aee-9070-051fc70f20b7 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:15.576662 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1ff71a-afe8-42af-bb52-88e3d0f607b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:15.577468 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1ff71a-afe8-42af-bb52-88e3d0f607b6 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:15.577750 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1ff71a-afe8-42af-bb52-88e3d0f607b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:15.578030 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1ff71a-afe8-42af-bb52-88e3d0f607b6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:15.587090 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1ff71a-afe8-42af-bb52-88e3d0f607b6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:15.588342 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1ff71a-afe8-42af-bb52-88e3d0f607b6 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:15.604370 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-1a1ff71a-afe8-42af-bb52-88e3d0f607b6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 21 18:07:15.604370 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:07:15.604370 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:07:15.604370 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:07:15.604370 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:07:15.604370 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:07:15.604370 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:07:15.604370 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:07:15.604370 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:07:15.604370 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:07:15.604370 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:07:15.604370 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:07:15.604370 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:07:15.604370 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:07:15.604370 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:07:15.604370 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:07:15.611256 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:07:15.611256 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:07:15.611256 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:07:15.611256 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:07:15.611256 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:07:15.611256 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:07:15.611256 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:07:15.611256 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:07:15.611256 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:07:15.611256 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:07:15.611256 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:07:15.611256 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:07:15.611256 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:07:15.611256 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:07:15.611256 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:07:15.611256 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:07:15.611256 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:07:15.611256 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:07:15.612247 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:07:15.612247 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:07:15.612247 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:07:15.612247 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:07:15.612247 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:07:15.612247 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:07:15.612247 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:07:15.612247 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:07:15.612247 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 21 18:07:15.612247 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:07:15.612247 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 540/1099] 10.222.0.23 () {56 vars in 1093 bytes} [Tue Jun 21 18:07:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:07:15.613053 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c96adb-54d5-4ace-9a99-15ba29a4928b None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:15.640472 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:15.723655 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c96adb-54d5-4ace-9a99-15ba29a4928b None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:15.766936 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99764359-27bb-4e16-bffb-e953aead538c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:15.768076 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99764359-27bb-4e16-bffb-e953aead538c None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:15.768629 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99764359-27bb-4e16-bffb-e953aead538c None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:15.769134 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99764359-27bb-4e16-bffb-e953aead538c None admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:15.770761 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99764359-27bb-4e16-bffb-e953aead538c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:07:15.771311 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99764359-27bb-4e16-bffb-e953aead538c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:15.774318 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99764359-27bb-4e16-bffb-e953aead538c None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:15.805613 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 560/1100] 10.222.0.23 () {56 vars in 1133 bytes} [Tue Jun 21 18:07:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:07:15.811527 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1ff71a-afe8-42af-bb52-88e3d0f607b6 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:15.847259 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:15.939032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1ff71a-afe8-42af-bb52-88e3d0f607b6 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:15.981089 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa54b9b-789b-4102-bd04-4c54b77c96b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:15.981889 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa54b9b-789b-4102-bd04-4c54b77c96b1 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:15.982274 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa54b9b-789b-4102-bd04-4c54b77c96b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:15.982651 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa54b9b-789b-4102-bd04-4c54b77c96b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:16.058978 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa54b9b-789b-4102-bd04-4c54b77c96b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:16.060462 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa54b9b-789b-4102-bd04-4c54b77c96b1 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:16.094679 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:16.175829 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 541/1101] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:07:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 18:07:16.183077 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99764359-27bb-4e16-bffb-e953aead538c None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:16.331577 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:16.364987 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99764359-27bb-4e16-bffb-e953aead538c None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:16.410698 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b51c3c-d5e9-45c6-99f6-8314d43cb3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:16.411656 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b51c3c-d5e9-45c6-99f6-8314d43cb3ef None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:16.412053 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b51c3c-d5e9-45c6-99f6-8314d43cb3ef None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:16.412469 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b51c3c-d5e9-45c6-99f6-8314d43cb3ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:16.424902 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b51c3c-d5e9-45c6-99f6-8314d43cb3ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:16.426523 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b51c3c-d5e9-45c6-99f6-8314d43cb3ef None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:16.448061 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-55b51c3c-d5e9-45c6-99f6-8314d43cb3ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:07:16.448061 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:07:16.448061 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:07:16.448061 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:07:16.448061 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:07:16.448061 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:07:16.448061 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:07:16.448061 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:07:16.448061 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:07:16.448061 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:07:16.448061 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:07:16.448061 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:07:16.448061 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:07:16.448061 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:07:16.448061 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:07:16.450177 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:07:16.450177 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:07:16.450177 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:07:16.450177 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:07:16.450177 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:07:16.450177 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:07:16.450177 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:07:16.450177 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:07:16.450177 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:07:16.450177 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:07:16.450177 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:07:16.450177 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:07:16.450177 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:07:16.450177 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:07:16.450177 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:07:16.450177 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:07:16.450177 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:07:16.452217 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:07:16.452217 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:07:16.452217 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:07:16.452217 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:07:16.456134 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 561/1102] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:07:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:07:16.458128 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa54b9b-789b-4102-bd04-4c54b77c96b1 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:16.591034 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa54b9b-789b-4102-bd04-4c54b77c96b1 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:16.646440 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:16.654732 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7052e8-7981-4e1b-a6b9-0e66743a1b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:16.655830 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7052e8-7981-4e1b-a6b9-0e66743a1b64 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:16.656296 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7052e8-7981-4e1b-a6b9-0e66743a1b64 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:16.656788 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7052e8-7981-4e1b-a6b9-0e66743a1b64 None admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:16.657992 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7052e8-7981-4e1b-a6b9-0e66743a1b64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:07:16.658415 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7052e8-7981-4e1b-a6b9-0e66743a1b64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:16.659828 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7052e8-7981-4e1b-a6b9-0e66743a1b64 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:16.675003 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 542/1103] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:07:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:07:16.682075 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b51c3c-d5e9-45c6-99f6-8314d43cb3ef None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:16.795698 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b51c3c-d5e9-45c6-99f6-8314d43cb3ef None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:16.834946 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757e87c7-e089-4458-acd8-47ac430bac97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:16.836837 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e87c7-e089-4458-acd8-47ac430bac97 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:16.837327 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e87c7-e089-4458-acd8-47ac430bac97 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:16.837782 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e87c7-e089-4458-acd8-47ac430bac97 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:16.841126 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:16.850757 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757e87c7-e089-4458-acd8-47ac430bac97 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:16.854553 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757e87c7-e089-4458-acd8-47ac430bac97 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:16.881691 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-757e87c7-e089-4458-acd8-47ac430bac97 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:07:16.881691 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:07:16.881691 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:07:16.881691 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:07:16.881691 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:07:16.881691 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:07:16.881691 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:07:16.881691 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:07:16.881691 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:07:16.881691 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:07:16.881691 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:07:16.881691 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:07:16.881691 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:07:16.881691 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:07:16.881691 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:07:16.883730 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:07:16.883730 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:07:16.883730 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:07:16.883730 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:07:16.883730 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:07:16.883730 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:07:16.883730 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:07:16.883730 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:07:16.883730 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:07:16.883730 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:07:16.883730 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:07:16.883730 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:07:16.883730 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:07:16.883730 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:07:16.883730 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:07:16.883730 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:07:16.883730 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:07:16.891620 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:07:16.891620 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:07:16.891620 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:07:16.891620 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:07:16.891620 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:07:16.891620 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:07:16.891620 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:07:16.891620 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:07:16.891620 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 562/1104] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jun 21 18:07:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:07:16.892862 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7052e8-7981-4e1b-a6b9-0e66743a1b64 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:17.017249 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7052e8-7981-4e1b-a6b9-0e66743a1b64 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:17.080099 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e737b5c6-d56a-4e85-b58b-8bf075e2ffd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:17.081222 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737b5c6-d56a-4e85-b58b-8bf075e2ffd5 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:17.081662 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737b5c6-d56a-4e85-b58b-8bf075e2ffd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:17.082109 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737b5c6-d56a-4e85-b58b-8bf075e2ffd5 None admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:17.083465 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e737b5c6-d56a-4e85-b58b-8bf075e2ffd5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:07:17.083980 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e737b5c6-d56a-4e85-b58b-8bf075e2ffd5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:17.087411 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e737b5c6-d56a-4e85-b58b-8bf075e2ffd5 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:17.113983 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 543/1105] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jun 21 18:07:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:07:17.115826 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757e87c7-e089-4458-acd8-47ac430bac97 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:17.123545 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b047a8-b50b-4a9c-989e-e03301efff65 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:17.123832 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b047a8-b50b-4a9c-989e-e03301efff65 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:17.124113 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b047a8-b50b-4a9c-989e-e03301efff65 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:17.125599 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 544/1106] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:07:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:07:17.131434 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd73c2b7-611d-4905-b58a-43c049f762ff None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:17.131726 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd73c2b7-611d-4905-b58a-43c049f762ff None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:17.132009 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd73c2b7-611d-4905-b58a-43c049f762ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:17.150569 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-fd73c2b7-611d-4905-b58a-43c049f762ff None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:07:17.151837 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-fd73c2b7-611d-4905-b58a-43c049f762ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:07:17.208448 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:17.236640 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 545/1107] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:07:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:07:17.248145 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757e87c7-e089-4458-acd8-47ac430bac97 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:17.263936 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd73c2b7-611d-4905-b58a-43c049f762ff None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:17.291030 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8246f0d7-70cb-4235-b528-bfd0fcf18567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:17.291904 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8246f0d7-70cb-4235-b528-bfd0fcf18567 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:17.292314 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8246f0d7-70cb-4235-b528-bfd0fcf18567 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:17.292688 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8246f0d7-70cb-4235-b528-bfd0fcf18567 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:17.293760 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8246f0d7-70cb-4235-b528-bfd0fcf18567 None admin] RBAC: Adding query filter params (role.id=e950e7aaae294af383fc9e83eed8250f, scope.project.id=2a4699d1ec70456da5f8e0aa4b1439b7, user.id=904255dd3c1f406eb10f6fc65f9cd098) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:07:17.294159 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8246f0d7-70cb-4235-b528-bfd0fcf18567 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:17.296112 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8246f0d7-70cb-4235-b528-bfd0fcf18567 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:17.303231 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 563/1108] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jun 21 18:07:17 2022] GET /identity/v3/role_assignments?user.id=904255dd3c1f406eb10f6fc65f9cd098&scope.project.id=2a4699d1ec70456da5f8e0aa4b1439b7&role.id=e950e7aaae294af383fc9e83eed8250f => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:07:17.377396 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd73c2b7-611d-4905-b58a-43c049f762ff None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:17.416581 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98a7376-0941-4804-a091-37e11bae1931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:17.417414 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98a7376-0941-4804-a091-37e11bae1931 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:17.417700 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98a7376-0941-4804-a091-37e11bae1931 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:17.417978 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98a7376-0941-4804-a091-37e11bae1931 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:17.419318 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98a7376-0941-4804-a091-37e11bae1931 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:17.420819 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98a7376-0941-4804-a091-37e11bae1931 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:17.422904 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 546/1109] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:07:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:07:17.428930 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8246f0d7-70cb-4235-b528-bfd0fcf18567 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:17.495548 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:17.543572 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8246f0d7-70cb-4235-b528-bfd0fcf18567 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:17.588750 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d060c130-63c5-4d44-a570-065fd39af2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:17.589859 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060c130-63c5-4d44-a570-065fd39af2d9 None admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:17.590349 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060c130-63c5-4d44-a570-065fd39af2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:17.590844 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060c130-63c5-4d44-a570-065fd39af2d9 None admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:17.592211 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d060c130-63c5-4d44-a570-065fd39af2d9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:17.594779 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d060c130-63c5-4d44-a570-065fd39af2d9 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:17.700839 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 564/1110] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Jun 21 18:07:17 2022] POST /identity/v3/users => generated 286 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:07:19.914689 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe49a8-984c-4dcd-81a6-446b8992d06c None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:19.915319 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe49a8-984c-4dcd-81a6-446b8992d06c None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:19.915319 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe49a8-984c-4dcd-81a6-446b8992d06c None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:19.916662 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 547/1111] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:07:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:07:19.925798 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce1ea5-276d-4d4c-aa94-3ae50c189c02 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:19.926342 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce1ea5-276d-4d4c-aa94-3ae50c189c02 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:19.926822 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce1ea5-276d-4d4c-aa94-3ae50c189c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:19.954780 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-9bce1ea5-276d-4d4c-aa94-3ae50c189c02 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:07:19.956796 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-9bce1ea5-276d-4d4c-aa94-3ae50c189c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:07:20.050110 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 565/1112] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:07:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:07:20.074691 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fe49a8-984c-4dcd-81a6-446b8992d06c None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:20.128336 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:20.188259 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fe49a8-984c-4dcd-81a6-446b8992d06c None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:20.228500 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e59287-c551-4ec4-98be-7fc3b01d9cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:20.229381 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e59287-c551-4ec4-98be-7fc3b01d9cf3 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:20.229671 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e59287-c551-4ec4-98be-7fc3b01d9cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:20.229951 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e59287-c551-4ec4-98be-7fc3b01d9cf3 None admin] PATH_INFO: `/v3/roles/482dcc9ba0fd400cad2d193e3e895d4c` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:20.231554 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e59287-c551-4ec4-98be-7fc3b01d9cf3 None admin] RBAC: Authorizing `identity:get_role(role_id=482dcc9ba0fd400cad2d193e3e895d4c)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:20.232990 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e59287-c551-4ec4-98be-7fc3b01d9cf3 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:20.234454 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 548/1113] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jun 21 18:07:20 2022] GET /identity/v3/roles/482dcc9ba0fd400cad2d193e3e895d4c => generated 223 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:07:20.240576 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bce1ea5-276d-4d4c-aa94-3ae50c189c02 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:20.291370 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:20.349187 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bce1ea5-276d-4d4c-aa94-3ae50c189c02 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:20.387035 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f18fcd-2361-4512-8133-bf10e963ca6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:20.387919 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f18fcd-2361-4512-8133-bf10e963ca6d None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:20.388206 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f18fcd-2361-4512-8133-bf10e963ca6d None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:20.388522 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f18fcd-2361-4512-8133-bf10e963ca6d None admin] PATH_INFO: `/v3/users/bf535c6aca13420d9d5e5587e8ff3c25` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:20.402009 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f18fcd-2361-4512-8133-bf10e963ca6d None admin] RBAC: Authorizing `identity:get_user(user_id=bf535c6aca13420d9d5e5587e8ff3c25)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:20.403472 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f18fcd-2361-4512-8133-bf10e963ca6d None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:20.405222 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 566/1114] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jun 21 18:07:20 2022] GET /identity/v3/users/bf535c6aca13420d9d5e5587e8ff3c25 => generated 286 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:07:20.411249 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e59287-c551-4ec4-98be-7fc3b01d9cf3 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:20.511877 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e59287-c551-4ec4-98be-7fc3b01d9cf3 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:20.529351 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:20.546829 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c802cb-54ff-4b93-bb44-d187138c0c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:20.547489 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c802cb-54ff-4b93-bb44-d187138c0c15 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:20.547775 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c802cb-54ff-4b93-bb44-d187138c0c15 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:20.548053 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c802cb-54ff-4b93-bb44-d187138c0c15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:20.617160 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c802cb-54ff-4b93-bb44-d187138c0c15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:20.618832 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c802cb-54ff-4b93-bb44-d187138c0c15 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:20.746131 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 549/1115] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:07:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 18:07:20.752302 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f18fcd-2361-4512-8133-bf10e963ca6d None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:20.861559 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f18fcd-2361-4512-8133-bf10e963ca6d None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:20.864324 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:20.900145 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a91a82b-95d2-4e8a-ad9d-de4878a301ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:20.901047 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a91a82b-95d2-4e8a-ad9d-de4878a301ce None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:20.901327 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a91a82b-95d2-4e8a-ad9d-de4878a301ce None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:20.901599 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a91a82b-95d2-4e8a-ad9d-de4878a301ce None admin] PATH_INFO: `/v3/projects/c7bd5615d37a425cb7bed3985d3eb711` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:20.902866 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a91a82b-95d2-4e8a-ad9d-de4878a301ce None admin] RBAC: Authorizing `identity:get_project(project_id=c7bd5615d37a425cb7bed3985d3eb711)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:20.904310 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a91a82b-95d2-4e8a-ad9d-de4878a301ce None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:20.906051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 567/1116] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jun 21 18:07:20 2022] GET /identity/v3/projects/c7bd5615d37a425cb7bed3985d3eb711 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:07:20.915095 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c802cb-54ff-4b93-bb44-d187138c0c15 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:20.935814 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:21.031461 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c802cb-54ff-4b93-bb44-d187138c0c15 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:21.092368 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8f100f-a211-4ffb-bdf0-5c003a0dd28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:21.093552 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8f100f-a211-4ffb-bdf0-5c003a0dd28d None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:21.094045 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8f100f-a211-4ffb-bdf0-5c003a0dd28d None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:21.094528 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8f100f-a211-4ffb-bdf0-5c003a0dd28d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:21.096113 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8f100f-a211-4ffb-bdf0-5c003a0dd28d None admin] RBAC: Adding query filter params (role.id=482dcc9ba0fd400cad2d193e3e895d4c, scope.project.id=c7bd5615d37a425cb7bed3985d3eb711, user.id=bf535c6aca13420d9d5e5587e8ff3c25) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:07:21.096730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8f100f-a211-4ffb-bdf0-5c003a0dd28d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:21.099520 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8f100f-a211-4ffb-bdf0-5c003a0dd28d None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:21.111001 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 550/1117] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jun 21 18:07:20 2022] GET /identity/v3/role_assignments?user.id=bf535c6aca13420d9d5e5587e8ff3c25&scope.project.id=c7bd5615d37a425cb7bed3985d3eb711&role.id=482dcc9ba0fd400cad2d193e3e895d4c => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:07:23.192075 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d548f472-2d45-49e7-90ed-9bc3d254f737 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:23.192751 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d548f472-2d45-49e7-90ed-9bc3d254f737 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:23.192881 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d548f472-2d45-49e7-90ed-9bc3d254f737 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:23.194277 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 568/1118] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:07:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:07:23.200701 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11faf7c8-f3c1-4318-bcec-d7a7638c693a None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:23.201230 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11faf7c8-f3c1-4318-bcec-d7a7638c693a None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:23.201230 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11faf7c8-f3c1-4318-bcec-d7a7638c693a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:23.218453 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-11faf7c8-f3c1-4318-bcec-d7a7638c693a None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:07:23.219672 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-11faf7c8-f3c1-4318-bcec-d7a7638c693a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:07:23.292708 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 551/1119] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:07:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:07:23.316418 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d548f472-2d45-49e7-90ed-9bc3d254f737 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:23.423018 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d548f472-2d45-49e7-90ed-9bc3d254f737 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:23.458731 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d69e0e2-a1fe-4074-ac55-805b29c1bced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:23.459570 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d69e0e2-a1fe-4074-ac55-805b29c1bced None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:23.459901 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d69e0e2-a1fe-4074-ac55-805b29c1bced None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:23.460227 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d69e0e2-a1fe-4074-ac55-805b29c1bced None admin] PATH_INFO: `/v3/roles/482dcc9ba0fd400cad2d193e3e895d4c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:23.462191 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d69e0e2-a1fe-4074-ac55-805b29c1bced None admin] RBAC: Authorizing `identity:get_role(role_id=482dcc9ba0fd400cad2d193e3e895d4c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:23.463643 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d69e0e2-a1fe-4074-ac55-805b29c1bced None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:23.465129 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 569/1120] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jun 21 18:07:23 2022] GET /identity/v3/roles/482dcc9ba0fd400cad2d193e3e895d4c => generated 223 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:07:23.471417 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11faf7c8-f3c1-4318-bcec-d7a7638c693a None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:23.487115 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:23.576291 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11faf7c8-f3c1-4318-bcec-d7a7638c693a None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:23.611171 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4344ada2-6a28-4f84-83c6-8dd3a702f846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:23.612048 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4344ada2-6a28-4f84-83c6-8dd3a702f846 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:23.612336 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4344ada2-6a28-4f84-83c6-8dd3a702f846 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:23.612645 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4344ada2-6a28-4f84-83c6-8dd3a702f846 None admin] PATH_INFO: `/v3/users/bf535c6aca13420d9d5e5587e8ff3c25` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:23.613855 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4344ada2-6a28-4f84-83c6-8dd3a702f846 None admin] RBAC: Authorizing `identity:get_user(user_id=bf535c6aca13420d9d5e5587e8ff3c25)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:23.615296 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4344ada2-6a28-4f84-83c6-8dd3a702f846 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:23.617000 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 552/1121] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jun 21 18:07:23 2022] GET /identity/v3/users/bf535c6aca13420d9d5e5587e8ff3c25 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:07:23.627203 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d69e0e2-a1fe-4074-ac55-805b29c1bced None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:23.697808 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:23.760627 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d69e0e2-a1fe-4074-ac55-805b29c1bced None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:23.796971 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3069cb26-a000-4d73-a8fe-1859fb45bf52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:23.797846 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069cb26-a000-4d73-a8fe-1859fb45bf52 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:23.798134 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069cb26-a000-4d73-a8fe-1859fb45bf52 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:23.798412 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069cb26-a000-4d73-a8fe-1859fb45bf52 None admin] PATH_INFO: `/v3/projects/c7bd5615d37a425cb7bed3985d3eb711` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:23.799628 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3069cb26-a000-4d73-a8fe-1859fb45bf52 None admin] RBAC: Authorizing `identity:get_project(project_id=c7bd5615d37a425cb7bed3985d3eb711)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:23.801111 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3069cb26-a000-4d73-a8fe-1859fb45bf52 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:23.802846 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 570/1122] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jun 21 18:07:23 2022] GET /identity/v3/projects/c7bd5615d37a425cb7bed3985d3eb711 => generated 311 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:07:23.809135 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4344ada2-6a28-4f84-83c6-8dd3a702f846 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:23.898328 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:23.918776 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4344ada2-6a28-4f84-83c6-8dd3a702f846 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:23.955479 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df60de6-4cdc-400c-896a-7bb49ea87906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:23.956382 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df60de6-4cdc-400c-896a-7bb49ea87906 None admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:23.956697 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df60de6-4cdc-400c-896a-7bb49ea87906 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:23.956977 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df60de6-4cdc-400c-896a-7bb49ea87906 None admin] PATH_INFO: `/v3/projects/c7bd5615d37a425cb7bed3985d3eb711/users/bf535c6aca13420d9d5e5587e8ff3c25/roles/482dcc9ba0fd400cad2d193e3e895d4c` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:23.959216 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df60de6-4cdc-400c-896a-7bb49ea87906 None admin] RBAC: Authorizing `identity:create_grant(project_id=c7bd5615d37a425cb7bed3985d3eb711, user_id=bf535c6aca13420d9d5e5587e8ff3c25, role_id=482dcc9ba0fd400cad2d193e3e895d4c)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:23.960704 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df60de6-4cdc-400c-896a-7bb49ea87906 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:24.025737 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 553/1123] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jun 21 18:07:23 2022] PUT /identity/v3/projects/c7bd5615d37a425cb7bed3985d3eb711/users/bf535c6aca13420d9d5e5587e8ff3c25/roles/482dcc9ba0fd400cad2d193e3e895d4c => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:07:26.121759 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a360e35-12b1-4d3d-a88d-3ff7c4c2318f None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:26.122511 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a360e35-12b1-4d3d-a88d-3ff7c4c2318f None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:26.122511 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a360e35-12b1-4d3d-a88d-3ff7c4c2318f None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:26.123707 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 571/1124] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:07:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:07:26.129521 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d692e6e-0931-4bdf-a549-2123b2f6a119 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:26.130028 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d692e6e-0931-4bdf-a549-2123b2f6a119 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:26.130120 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d692e6e-0931-4bdf-a549-2123b2f6a119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:26.147878 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-4d692e6e-0931-4bdf-a549-2123b2f6a119 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:07:26.149147 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-4d692e6e-0931-4bdf-a549-2123b2f6a119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:07:26.223410 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 554/1125] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:07:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:07:26.248974 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a360e35-12b1-4d3d-a88d-3ff7c4c2318f None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:26.390720 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a360e35-12b1-4d3d-a88d-3ff7c4c2318f None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:26.405875 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:26.441997 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a30a8d-95ca-40d9-ac6f-61a5f2ea9fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:26.442975 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a30a8d-95ca-40d9-ac6f-61a5f2ea9fe9 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:26.443260 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a30a8d-95ca-40d9-ac6f-61a5f2ea9fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:26.443539 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a30a8d-95ca-40d9-ac6f-61a5f2ea9fe9 None admin] PATH_INFO: `/v3/roles/482dcc9ba0fd400cad2d193e3e895d4c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:26.445142 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a30a8d-95ca-40d9-ac6f-61a5f2ea9fe9 None admin] RBAC: Authorizing `identity:get_role(role_id=482dcc9ba0fd400cad2d193e3e895d4c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:26.447146 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a30a8d-95ca-40d9-ac6f-61a5f2ea9fe9 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:26.448996 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 572/1126] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jun 21 18:07:26 2022] GET /identity/v3/roles/482dcc9ba0fd400cad2d193e3e895d4c => generated 223 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:07:26.458038 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d692e6e-0931-4bdf-a549-2123b2f6a119 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:26.576230 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:26.630822 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d692e6e-0931-4bdf-a549-2123b2f6a119 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:26.676526 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f777b4b1-5446-4a45-894d-a1426fd57a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:26.677335 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f777b4b1-5446-4a45-894d-a1426fd57a43 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:26.677621 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f777b4b1-5446-4a45-894d-a1426fd57a43 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:26.677901 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f777b4b1-5446-4a45-894d-a1426fd57a43 None admin] PATH_INFO: `/v3/users/bf535c6aca13420d9d5e5587e8ff3c25` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:26.679173 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f777b4b1-5446-4a45-894d-a1426fd57a43 None admin] RBAC: Authorizing `identity:get_user(user_id=bf535c6aca13420d9d5e5587e8ff3c25)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:26.681566 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f777b4b1-5446-4a45-894d-a1426fd57a43 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:26.683320 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 555/1127] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jun 21 18:07:26 2022] GET /identity/v3/users/bf535c6aca13420d9d5e5587e8ff3c25 => generated 286 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:07:26.689484 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a30a8d-95ca-40d9-ac6f-61a5f2ea9fe9 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:26.800186 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a30a8d-95ca-40d9-ac6f-61a5f2ea9fe9 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:26.838440 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef25bc6e-8291-48ec-90a4-5ace78c4c13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:26.839112 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25bc6e-8291-48ec-90a4-5ace78c4c13f None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:26.839400 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25bc6e-8291-48ec-90a4-5ace78c4c13f None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:26.839684 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25bc6e-8291-48ec-90a4-5ace78c4c13f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:26.893679 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:26.936820 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef25bc6e-8291-48ec-90a4-5ace78c4c13f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:26.939468 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef25bc6e-8291-48ec-90a4-5ace78c4c13f None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:27.069927 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 573/1128] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:07:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 18:07:27.076848 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f777b4b1-5446-4a45-894d-a1426fd57a43 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:27.192041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f777b4b1-5446-4a45-894d-a1426fd57a43 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:27.228823 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2c6d8a-f629-4dbf-b9c8-d7a2d33c5f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:27.229704 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c6d8a-f629-4dbf-b9c8-d7a2d33c5f2d None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:27.229985 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c6d8a-f629-4dbf-b9c8-d7a2d33c5f2d None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:27.230260 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c6d8a-f629-4dbf-b9c8-d7a2d33c5f2d None admin] PATH_INFO: `/v3/projects/c7bd5615d37a425cb7bed3985d3eb711` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:27.231480 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2c6d8a-f629-4dbf-b9c8-d7a2d33c5f2d None admin] RBAC: Authorizing `identity:get_project(project_id=c7bd5615d37a425cb7bed3985d3eb711)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:27.232959 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2c6d8a-f629-4dbf-b9c8-d7a2d33c5f2d None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:27.234700 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 556/1129] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jun 21 18:07:27 2022] GET /identity/v3/projects/c7bd5615d37a425cb7bed3985d3eb711 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:07:27.241077 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef25bc6e-8291-48ec-90a4-5ace78c4c13f None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:27.322797 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:27.348324 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef25bc6e-8291-48ec-90a4-5ace78c4c13f None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:27.388508 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fc0f43-be9b-4117-878a-f8659f2b3658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:27.389178 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc0f43-be9b-4117-878a-f8659f2b3658 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:27.389465 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc0f43-be9b-4117-878a-f8659f2b3658 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:27.389744 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc0f43-be9b-4117-878a-f8659f2b3658 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:27.390595 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fc0f43-be9b-4117-878a-f8659f2b3658 None admin] RBAC: Adding query filter params (role.id=482dcc9ba0fd400cad2d193e3e895d4c, scope.project.id=c7bd5615d37a425cb7bed3985d3eb711, user.id=bf535c6aca13420d9d5e5587e8ff3c25) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:07:27.390909 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fc0f43-be9b-4117-878a-f8659f2b3658 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:27.392365 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fc0f43-be9b-4117-878a-f8659f2b3658 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:27.399363 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 574/1130] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jun 21 18:07:27 2022] GET /identity/v3/role_assignments?user.id=bf535c6aca13420d9d5e5587e8ff3c25&scope.project.id=c7bd5615d37a425cb7bed3985d3eb711&role.id=482dcc9ba0fd400cad2d193e3e895d4c => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:07:29.578783 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311898c7-00f6-488f-be90-63b4d7050418 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:29.586920 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311898c7-00f6-488f-be90-63b4d7050418 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:29.586920 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311898c7-00f6-488f-be90-63b4d7050418 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:29.586920 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 557/1131] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:07:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:07:29.587585 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade2cc8f-d204-4a7d-955e-39a7ba6c1d71 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:29.588055 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade2cc8f-d204-4a7d-955e-39a7ba6c1d71 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:29.588237 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade2cc8f-d204-4a7d-955e-39a7ba6c1d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:29.606370 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-ade2cc8f-d204-4a7d-955e-39a7ba6c1d71 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:07:29.607598 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-ade2cc8f-d204-4a7d-955e-39a7ba6c1d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:07:29.683702 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 575/1132] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:07:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:07:29.709049 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311898c7-00f6-488f-be90-63b4d7050418 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:29.720357 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:29.827142 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311898c7-00f6-488f-be90-63b4d7050418 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:29.866033 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89b6f63-7233-428a-acf6-56f551ed813b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:29.866748 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b6f63-7233-428a-acf6-56f551ed813b None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:29.867032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b6f63-7233-428a-acf6-56f551ed813b None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:29.867870 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b6f63-7233-428a-acf6-56f551ed813b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:29.869265 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89b6f63-7233-428a-acf6-56f551ed813b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:29.870701 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89b6f63-7233-428a-acf6-56f551ed813b None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:29.872361 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 558/1133] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:07:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:07:29.881612 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade2cc8f-d204-4a7d-955e-39a7ba6c1d71 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:29.956694 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:29.993785 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade2cc8f-d204-4a7d-955e-39a7ba6c1d71 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:30.031003 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc305ff-f328-40b3-af56-5b1ccfd61836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:30.031682 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc305ff-f328-40b3-af56-5b1ccfd61836 None admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:30.031968 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc305ff-f328-40b3-af56-5b1ccfd61836 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:30.032246 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc305ff-f328-40b3-af56-5b1ccfd61836 None admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:30.033020 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc305ff-f328-40b3-af56-5b1ccfd61836 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:30.034472 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc305ff-f328-40b3-af56-5b1ccfd61836 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:30.121085 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 576/1134] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Jun 21 18:07:29 2022] POST /identity/v3/projects => generated 311 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:07:32.275546 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da1f885-d315-42f9-829b-d59692499961 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:32.276181 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da1f885-d315-42f9-829b-d59692499961 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:32.276181 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da1f885-d315-42f9-829b-d59692499961 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:32.277618 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 559/1135] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:07:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:07:32.287589 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51336b67-544a-49ed-b55b-3464dc064e33 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:32.288183 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51336b67-544a-49ed-b55b-3464dc064e33 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:32.288596 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51336b67-544a-49ed-b55b-3464dc064e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:32.319063 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-51336b67-544a-49ed-b55b-3464dc064e33 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:07:32.324336 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-51336b67-544a-49ed-b55b-3464dc064e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:07:32.450553 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 577/1136] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:07:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:07:32.477920 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da1f885-d315-42f9-829b-d59692499961 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:32.562686 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:32.591066 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da1f885-d315-42f9-829b-d59692499961 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:32.627576 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212eeb73-e128-463d-972e-668d5f72eff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:32.628380 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212eeb73-e128-463d-972e-668d5f72eff9 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:32.628690 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212eeb73-e128-463d-972e-668d5f72eff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:32.628966 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212eeb73-e128-463d-972e-668d5f72eff9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:32.630210 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212eeb73-e128-463d-972e-668d5f72eff9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:32.631648 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212eeb73-e128-463d-972e-668d5f72eff9 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:32.634583 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 560/1137] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:07:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:07:32.640673 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51336b67-544a-49ed-b55b-3464dc064e33 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:32.694500 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:32.749071 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51336b67-544a-49ed-b55b-3464dc064e33 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:32.787178 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7116518-6832-4d2f-843a-8e3ba9922a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:32.787865 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7116518-6832-4d2f-843a-8e3ba9922a46 None admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:32.788189 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7116518-6832-4d2f-843a-8e3ba9922a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:32.788455 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7116518-6832-4d2f-843a-8e3ba9922a46 None admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:32.789308 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7116518-6832-4d2f-843a-8e3ba9922a46 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:32.790862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7116518-6832-4d2f-843a-8e3ba9922a46 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:32.899336 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 578/1138] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Jun 21 18:07:32 2022] POST /identity/v3/users => generated 286 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:07:35.076530 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d4d573-18f1-415b-aab3-19a2e698a067 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:35.077413 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d4d573-18f1-415b-aab3-19a2e698a067 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:35.077413 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d4d573-18f1-415b-aab3-19a2e698a067 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:35.078548 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 561/1139] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:07:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:07:35.084998 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d4680-cbaa-47c8-8165-67bd3ec5d3f8 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:35.085918 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d4680-cbaa-47c8-8165-67bd3ec5d3f8 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:35.085918 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d4680-cbaa-47c8-8165-67bd3ec5d3f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:35.104620 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-001d4680-cbaa-47c8-8165-67bd3ec5d3f8 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:07:35.105906 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-001d4680-cbaa-47c8-8165-67bd3ec5d3f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:07:35.180502 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 579/1140] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:07:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:07:35.205423 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d4d573-18f1-415b-aab3-19a2e698a067 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:35.314732 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d4d573-18f1-415b-aab3-19a2e698a067 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:35.351330 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7b1c46-aaa8-4fce-9740-ed75bbb7daec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:35.352207 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b1c46-aaa8-4fce-9740-ed75bbb7daec None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:35.352524 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b1c46-aaa8-4fce-9740-ed75bbb7daec None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:35.352807 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b1c46-aaa8-4fce-9740-ed75bbb7daec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:35.364502 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7b1c46-aaa8-4fce-9740-ed75bbb7daec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:07:35.372981 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-ed7b1c46-aaa8-4fce-9740-ed75bbb7daec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:07:35.372981 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:07:35.372981 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:07:35.372981 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:07:35.372981 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:07:35.372981 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:07:35.372981 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:07:35.372981 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:07:35.372981 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:07:35.372981 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:07:35.372981 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:07:35.372981 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:07:35.372981 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:07:35.372981 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:07:35.372981 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:07:35.372981 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:07:35.374877 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:07:35.374877 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:07:35.374877 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:07:35.374877 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:07:35.374877 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:07:35.374877 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:07:35.374877 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:07:35.374877 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:07:35.374877 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:07:35.374877 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:07:35.374877 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:07:35.374877 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:07:35.374877 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:07:35.374877 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:07:35.374877 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:07:35.374877 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:07:35.376839 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:07:35.376839 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:07:35.376839 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:07:35.376839 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:07:35.376839 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:07:35.376839 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:07:35.376839 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:07:35.376839 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:07:35.376839 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:07:35.376839 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:07:35.376839 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:07:35.376839 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:07:35.376839 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 562/1141] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jun 21 18:07:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:07:35.382238 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001d4680-cbaa-47c8-8165-67bd3ec5d3f8 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:35.420850 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:35.491452 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001d4680-cbaa-47c8-8165-67bd3ec5d3f8 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:35.528570 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4b13bf-f486-4ca9-afa7-2ac215089b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:35.529247 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4b13bf-f486-4ca9-afa7-2ac215089b00 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:35.529526 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4b13bf-f486-4ca9-afa7-2ac215089b00 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:35.529797 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4b13bf-f486-4ca9-afa7-2ac215089b00 None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:35.530550 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4b13bf-f486-4ca9-afa7-2ac215089b00 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:07:35.530855 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4b13bf-f486-4ca9-afa7-2ac215089b00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:35.532223 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4b13bf-f486-4ca9-afa7-2ac215089b00 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:35.542224 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 580/1142] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:07:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:07:35.548146 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7b1c46-aaa8-4fce-9740-ed75bbb7daec None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:35.551203 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:35.654728 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7b1c46-aaa8-4fce-9740-ed75bbb7daec None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:35.691254 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb04dadd-2d7e-4a36-ab39-7b68d9f88586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:35.692142 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04dadd-2d7e-4a36-ab39-7b68d9f88586 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:35.692454 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04dadd-2d7e-4a36-ab39-7b68d9f88586 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:35.692742 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04dadd-2d7e-4a36-ab39-7b68d9f88586 None admin] PATH_INFO: `/v3/users/b0458c136e6a498b9a1758cb00399445` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:35.708961 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb04dadd-2d7e-4a36-ab39-7b68d9f88586 None admin] RBAC: Authorizing `identity:get_user(user_id=b0458c136e6a498b9a1758cb00399445)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:35.710556 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb04dadd-2d7e-4a36-ab39-7b68d9f88586 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:35.713016 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 563/1143] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jun 21 18:07:35 2022] GET /identity/v3/users/b0458c136e6a498b9a1758cb00399445 => generated 286 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:07:35.719103 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4b13bf-f486-4ca9-afa7-2ac215089b00 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:35.845335 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:35.849726 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4b13bf-f486-4ca9-afa7-2ac215089b00 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:35.888825 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68012d78-8879-4ca0-8f5d-ffacd881e74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:35.889542 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68012d78-8879-4ca0-8f5d-ffacd881e74e None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:35.889828 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68012d78-8879-4ca0-8f5d-ffacd881e74e None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:35.890110 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68012d78-8879-4ca0-8f5d-ffacd881e74e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:35.961977 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68012d78-8879-4ca0-8f5d-ffacd881e74e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:35.963458 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68012d78-8879-4ca0-8f5d-ffacd881e74e None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:36.063749 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:36.068410 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 581/1144] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:07:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 18:07:36.075692 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb04dadd-2d7e-4a36-ab39-7b68d9f88586 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:36.140143 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:36.185826 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb04dadd-2d7e-4a36-ab39-7b68d9f88586 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:36.220811 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8197ba-594b-44fc-9189-74ab6bbccacc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:36.221694 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8197ba-594b-44fc-9189-74ab6bbccacc None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:36.221980 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8197ba-594b-44fc-9189-74ab6bbccacc None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:36.222253 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8197ba-594b-44fc-9189-74ab6bbccacc None admin] PATH_INFO: `/v3/projects/317500a01e4a44aa9868cbc68db4fba3` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:36.223464 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8197ba-594b-44fc-9189-74ab6bbccacc None admin] RBAC: Authorizing `identity:get_project(project_id=317500a01e4a44aa9868cbc68db4fba3)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:36.225204 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8197ba-594b-44fc-9189-74ab6bbccacc None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:36.227538 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 564/1145] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jun 21 18:07:36 2022] GET /identity/v3/projects/317500a01e4a44aa9868cbc68db4fba3 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:07:36.233815 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68012d78-8879-4ca0-8f5d-ffacd881e74e None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:36.340138 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68012d78-8879-4ca0-8f5d-ffacd881e74e None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:36.375967 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de20c5b-fd5f-4827-807c-755eaddc9b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:36.376658 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de20c5b-fd5f-4827-807c-755eaddc9b09 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:36.376945 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de20c5b-fd5f-4827-807c-755eaddc9b09 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:36.377223 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de20c5b-fd5f-4827-807c-755eaddc9b09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:36.378057 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de20c5b-fd5f-4827-807c-755eaddc9b09 None admin] RBAC: Adding query filter params (role.id=745ed9065a8143e384ebbd694b29635c, scope.project.id=317500a01e4a44aa9868cbc68db4fba3, user.id=b0458c136e6a498b9a1758cb00399445) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:07:36.378368 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de20c5b-fd5f-4827-807c-755eaddc9b09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:36.379799 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de20c5b-fd5f-4827-807c-755eaddc9b09 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:36.386397 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 582/1146] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jun 21 18:07:36 2022] GET /identity/v3/role_assignments?user.id=b0458c136e6a498b9a1758cb00399445&scope.project.id=317500a01e4a44aa9868cbc68db4fba3&role.id=745ed9065a8143e384ebbd694b29635c => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:07:38.487790 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd00867-50f2-4c36-8177-3aad255e3725 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:38.488358 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd00867-50f2-4c36-8177-3aad255e3725 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:38.488358 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd00867-50f2-4c36-8177-3aad255e3725 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:38.489748 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 565/1147] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:07:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:07:38.495409 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275c7ca-309b-4627-a02c-9ae3471cc9b1 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:38.495891 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275c7ca-309b-4627-a02c-9ae3471cc9b1 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:38.495978 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275c7ca-309b-4627-a02c-9ae3471cc9b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:38.514646 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-f275c7ca-309b-4627-a02c-9ae3471cc9b1 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:07:38.515839 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-f275c7ca-309b-4627-a02c-9ae3471cc9b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:07:38.575095 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:38.585475 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 583/1148] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:07:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:07:38.609206 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd00867-50f2-4c36-8177-3aad255e3725 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:38.718212 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd00867-50f2-4c36-8177-3aad255e3725 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:38.746327 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:38.754305 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a03389a-ec55-4da5-9942-aaa08fc0ec8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:38.758741 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03389a-ec55-4da5-9942-aaa08fc0ec8d None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:38.758741 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03389a-ec55-4da5-9942-aaa08fc0ec8d None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:38.758741 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03389a-ec55-4da5-9942-aaa08fc0ec8d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:38.767217 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a03389a-ec55-4da5-9942-aaa08fc0ec8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:07:38.775606 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-7a03389a-ec55-4da5-9942-aaa08fc0ec8d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:07:38.775606 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:07:38.775606 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:07:38.775606 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:07:38.775606 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:07:38.775606 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:07:38.775606 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:07:38.775606 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:07:38.775606 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:07:38.775606 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:07:38.775606 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:07:38.775606 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:07:38.775606 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:07:38.775606 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:07:38.775606 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:07:38.775606 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:07:38.777317 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:07:38.777317 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:07:38.777317 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:07:38.777317 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:07:38.777317 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:07:38.777317 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:07:38.777317 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:07:38.777317 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:07:38.777317 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:07:38.777317 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:07:38.777317 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:07:38.777317 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:07:38.777317 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:07:38.777317 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:07:38.777317 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:07:38.777317 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:07:38.778236 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:07:38.778236 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:07:38.778236 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:07:38.778236 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:07:38.778236 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:07:38.778236 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:07:38.778236 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:07:38.778236 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:07:38.778236 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:07:38.778236 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:07:38.778236 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:07:38.778236 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:07:38.778236 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 566/1149] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jun 21 18:07:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:07:38.783143 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f275c7ca-309b-4627-a02c-9ae3471cc9b1 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:38.894187 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f275c7ca-309b-4627-a02c-9ae3471cc9b1 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:38.932571 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a53df9-2c7a-40f5-add0-2af310d9f95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:38.933253 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a53df9-2c7a-40f5-add0-2af310d9f95a None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:38.933541 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a53df9-2c7a-40f5-add0-2af310d9f95a None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:38.933822 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a53df9-2c7a-40f5-add0-2af310d9f95a None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:38.934584 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a53df9-2c7a-40f5-add0-2af310d9f95a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:07:38.934896 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a53df9-2c7a-40f5-add0-2af310d9f95a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:38.936274 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a53df9-2c7a-40f5-add0-2af310d9f95a None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:38.946117 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 584/1150] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:07:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:07:38.952149 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a03389a-ec55-4da5-9942-aaa08fc0ec8d None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:39.061103 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a03389a-ec55-4da5-9942-aaa08fc0ec8d None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:39.095517 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afc68aa-47f2-4b3f-9f82-f05af088df48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:39.096432 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc68aa-47f2-4b3f-9f82-f05af088df48 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:39.096716 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc68aa-47f2-4b3f-9f82-f05af088df48 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:39.096994 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc68aa-47f2-4b3f-9f82-f05af088df48 None admin] PATH_INFO: `/v3/users/b0458c136e6a498b9a1758cb00399445` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:39.098263 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afc68aa-47f2-4b3f-9f82-f05af088df48 None admin] RBAC: Authorizing `identity:get_user(user_id=b0458c136e6a498b9a1758cb00399445)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:39.099723 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afc68aa-47f2-4b3f-9f82-f05af088df48 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:39.102321 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 567/1151] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jun 21 18:07:38 2022] GET /identity/v3/users/b0458c136e6a498b9a1758cb00399445 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:07:39.108864 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a53df9-2c7a-40f5-add0-2af310d9f95a None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:39.207902 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:39.284261 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a53df9-2c7a-40f5-add0-2af310d9f95a None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:39.328110 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b98d29-a09e-4562-9c75-fa612aafb1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:39.328918 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b98d29-a09e-4562-9c75-fa612aafb1f3 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:39.329200 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b98d29-a09e-4562-9c75-fa612aafb1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:39.329476 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b98d29-a09e-4562-9c75-fa612aafb1f3 None admin] PATH_INFO: `/v3/projects/317500a01e4a44aa9868cbc68db4fba3` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:39.330670 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b98d29-a09e-4562-9c75-fa612aafb1f3 None admin] RBAC: Authorizing `identity:get_project(project_id=317500a01e4a44aa9868cbc68db4fba3)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:39.332105 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b98d29-a09e-4562-9c75-fa612aafb1f3 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:39.333967 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 585/1152] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jun 21 18:07:39 2022] GET /identity/v3/projects/317500a01e4a44aa9868cbc68db4fba3 => generated 311 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:07:39.340247 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afc68aa-47f2-4b3f-9f82-f05af088df48 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:39.400207 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:39.446382 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afc68aa-47f2-4b3f-9f82-f05af088df48 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:39.483623 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab9ab7a-b67b-43ed-9a7c-4931fa71d7a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:39.484621 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab9ab7a-b67b-43ed-9a7c-4931fa71d7a1 None admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:39.484944 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab9ab7a-b67b-43ed-9a7c-4931fa71d7a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:39.485260 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab9ab7a-b67b-43ed-9a7c-4931fa71d7a1 None admin] PATH_INFO: `/v3/projects/317500a01e4a44aa9868cbc68db4fba3/users/b0458c136e6a498b9a1758cb00399445/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:39.487342 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab9ab7a-b67b-43ed-9a7c-4931fa71d7a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=317500a01e4a44aa9868cbc68db4fba3, user_id=b0458c136e6a498b9a1758cb00399445, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:39.488865 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab9ab7a-b67b-43ed-9a7c-4931fa71d7a1 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:39.591918 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 568/1153] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jun 21 18:07:39 2022] PUT /identity/v3/projects/317500a01e4a44aa9868cbc68db4fba3/users/b0458c136e6a498b9a1758cb00399445/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:07:41.738002 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561b97fc-d29a-4913-a507-57f4648a6c7c None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:41.738615 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561b97fc-d29a-4913-a507-57f4648a6c7c None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:41.738615 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561b97fc-d29a-4913-a507-57f4648a6c7c None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:41.739965 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 586/1154] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:07:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:07:41.746776 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353237c-8c23-44b0-a33e-abf51885fb9a None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:41.747374 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353237c-8c23-44b0-a33e-abf51885fb9a None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:41.747374 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353237c-8c23-44b0-a33e-abf51885fb9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:41.766390 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-b353237c-8c23-44b0-a33e-abf51885fb9a None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:07:41.767724 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-b353237c-8c23-44b0-a33e-abf51885fb9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:07:41.841911 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 569/1155] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:07:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:07:41.870328 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561b97fc-d29a-4913-a507-57f4648a6c7c None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:41.987368 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561b97fc-d29a-4913-a507-57f4648a6c7c None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:42.016579 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:42.030120 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c237e45-5afe-4ef2-ad89-166ca893607b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:42.033438 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c237e45-5afe-4ef2-ad89-166ca893607b None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:42.033438 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c237e45-5afe-4ef2-ad89-166ca893607b None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:42.033438 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c237e45-5afe-4ef2-ad89-166ca893607b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:42.045144 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c237e45-5afe-4ef2-ad89-166ca893607b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:07:42.053598 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-6c237e45-5afe-4ef2-ad89-166ca893607b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:07:42.053598 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:07:42.053598 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:07:42.053598 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:07:42.053598 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:07:42.053598 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:07:42.053598 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:07:42.053598 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:07:42.053598 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:07:42.053598 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:07:42.053598 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:07:42.053598 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:07:42.053598 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:07:42.053598 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:07:42.053598 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:07:42.053598 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:07:42.055865 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:07:42.055865 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:07:42.055865 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:07:42.055865 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:07:42.055865 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:07:42.055865 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:07:42.055865 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:07:42.055865 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:07:42.055865 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:07:42.055865 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:07:42.055865 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:07:42.055865 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:07:42.055865 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:07:42.055865 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:07:42.055865 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:07:42.055865 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:07:42.057995 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:07:42.057995 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:07:42.057995 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:07:42.057995 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:07:42.057995 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:07:42.057995 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:07:42.057995 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:07:42.057995 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:07:42.057995 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:07:42.057995 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:07:42.057995 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:07:42.057995 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:07:42.057995 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 587/1156] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jun 21 18:07:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:07:42.063279 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b353237c-8c23-44b0-a33e-abf51885fb9a None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:42.119079 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:42.175461 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b353237c-8c23-44b0-a33e-abf51885fb9a None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:42.213238 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa01e2c-9187-48d4-a59c-6efc41676567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:42.213911 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa01e2c-9187-48d4-a59c-6efc41676567 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:42.214198 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa01e2c-9187-48d4-a59c-6efc41676567 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:42.214478 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa01e2c-9187-48d4-a59c-6efc41676567 None admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:42.215271 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa01e2c-9187-48d4-a59c-6efc41676567 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:07:42.215584 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa01e2c-9187-48d4-a59c-6efc41676567 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:42.217044 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa01e2c-9187-48d4-a59c-6efc41676567 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:42.227506 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 570/1157] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:07:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:07:42.233699 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c237e45-5afe-4ef2-ad89-166ca893607b None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:42.343130 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c237e45-5afe-4ef2-ad89-166ca893607b None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:42.381834 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909bdcae-88d8-40f0-85c6-1b58e95444d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:42.382832 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909bdcae-88d8-40f0-85c6-1b58e95444d0 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:42.383120 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909bdcae-88d8-40f0-85c6-1b58e95444d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:42.383396 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909bdcae-88d8-40f0-85c6-1b58e95444d0 None admin] PATH_INFO: `/v3/users/b0458c136e6a498b9a1758cb00399445` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:42.384790 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909bdcae-88d8-40f0-85c6-1b58e95444d0 None admin] RBAC: Authorizing `identity:get_user(user_id=b0458c136e6a498b9a1758cb00399445)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:42.386246 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909bdcae-88d8-40f0-85c6-1b58e95444d0 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:42.387933 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 588/1158] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jun 21 18:07:42 2022] GET /identity/v3/users/b0458c136e6a498b9a1758cb00399445 => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:07:42.394091 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa01e2c-9187-48d4-a59c-6efc41676567 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:42.514567 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa01e2c-9187-48d4-a59c-6efc41676567 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:42.525755 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:42.554207 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfc144b-8d34-4476-81fa-d7186614f64b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:42.555423 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc144b-8d34-4476-81fa-d7186614f64b None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:42.555710 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc144b-8d34-4476-81fa-d7186614f64b None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:42.555991 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc144b-8d34-4476-81fa-d7186614f64b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:42.632992 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfc144b-8d34-4476-81fa-d7186614f64b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:42.634534 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfc144b-8d34-4476-81fa-d7186614f64b None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:42.743445 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 571/1159] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:07:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 18:07:42.749808 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909bdcae-88d8-40f0-85c6-1b58e95444d0 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:42.807602 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:42.864546 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909bdcae-88d8-40f0-85c6-1b58e95444d0 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:42.928233 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a698f3e2-3e7a-4497-b518-0bb49c45e1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:42.930415 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a698f3e2-3e7a-4497-b518-0bb49c45e1e8 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:42.931792 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a698f3e2-3e7a-4497-b518-0bb49c45e1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:42.932272 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a698f3e2-3e7a-4497-b518-0bb49c45e1e8 None admin] PATH_INFO: `/v3/projects/317500a01e4a44aa9868cbc68db4fba3` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:42.934434 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a698f3e2-3e7a-4497-b518-0bb49c45e1e8 None admin] RBAC: Authorizing `identity:get_project(project_id=317500a01e4a44aa9868cbc68db4fba3)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:42.937202 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a698f3e2-3e7a-4497-b518-0bb49c45e1e8 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:42.940477 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 589/1160] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jun 21 18:07:42 2022] GET /identity/v3/projects/317500a01e4a44aa9868cbc68db4fba3 => generated 311 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:07:42.947638 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfc144b-8d34-4476-81fa-d7186614f64b None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:42.958579 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:43.064941 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfc144b-8d34-4476-81fa-d7186614f64b None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:43.105596 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf5afdc-4c5f-4190-9f9b-15101e8a6006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:43.106290 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf5afdc-4c5f-4190-9f9b-15101e8a6006 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:43.106578 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf5afdc-4c5f-4190-9f9b-15101e8a6006 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:43.106854 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf5afdc-4c5f-4190-9f9b-15101e8a6006 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:43.107703 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf5afdc-4c5f-4190-9f9b-15101e8a6006 None admin] RBAC: Adding query filter params (role.id=745ed9065a8143e384ebbd694b29635c, scope.project.id=317500a01e4a44aa9868cbc68db4fba3, user.id=b0458c136e6a498b9a1758cb00399445) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:07:43.108019 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf5afdc-4c5f-4190-9f9b-15101e8a6006 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:43.109544 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf5afdc-4c5f-4190-9f9b-15101e8a6006 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:43.116549 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 572/1161] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jun 21 18:07:42 2022] GET /identity/v3/role_assignments?user.id=b0458c136e6a498b9a1758cb00399445&scope.project.id=317500a01e4a44aa9868cbc68db4fba3&role.id=745ed9065a8143e384ebbd694b29635c => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:07:45.240172 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3271ea-4f46-4800-87d3-dbffd1b5a0c2 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:45.240920 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3271ea-4f46-4800-87d3-dbffd1b5a0c2 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:45.240920 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3271ea-4f46-4800-87d3-dbffd1b5a0c2 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:45.242301 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 590/1162] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:07:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:07:45.251627 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f1910-8a7c-417a-9186-6ed541ea93f4 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:45.252161 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f1910-8a7c-417a-9186-6ed541ea93f4 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:45.252621 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f1910-8a7c-417a-9186-6ed541ea93f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:45.283272 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-565f1910-8a7c-417a-9186-6ed541ea93f4 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:07:45.285443 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-565f1910-8a7c-417a-9186-6ed541ea93f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:07:45.411681 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 573/1163] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:07:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:07:45.435526 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3271ea-4f46-4800-87d3-dbffd1b5a0c2 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:45.546791 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3271ea-4f46-4800-87d3-dbffd1b5a0c2 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:45.567159 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:45.584950 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1763859a-5330-4e91-a4d9-f5b8fd5c0cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:45.586271 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1763859a-5330-4e91-a4d9-f5b8fd5c0cd7 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:45.586569 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1763859a-5330-4e91-a4d9-f5b8fd5c0cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:45.586845 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1763859a-5330-4e91-a4d9-f5b8fd5c0cd7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:45.661736 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1763859a-5330-4e91-a4d9-f5b8fd5c0cd7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:45.663260 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1763859a-5330-4e91-a4d9-f5b8fd5c0cd7 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:45.779438 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 591/1164] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:07:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 18:07:45.786004 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565f1910-8a7c-417a-9186-6ed541ea93f4 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:45.788669 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:45.895162 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565f1910-8a7c-417a-9186-6ed541ea93f4 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:45.931215 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976af8fe-0ccd-4206-bc9b-f7f39cea4ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:45.932021 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976af8fe-0ccd-4206-bc9b-f7f39cea4ad8 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:45.932307 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976af8fe-0ccd-4206-bc9b-f7f39cea4ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:45.932612 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976af8fe-0ccd-4206-bc9b-f7f39cea4ad8 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:45.939871 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976af8fe-0ccd-4206-bc9b-f7f39cea4ad8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:45.941376 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976af8fe-0ccd-4206-bc9b-f7f39cea4ad8 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:45.956720 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-976af8fe-0ccd-4206-bc9b-f7f39cea4ad8 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 21 18:07:45.956720 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:07:45.956720 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 21 18:07:45.956720 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 21 18:07:45.956720 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:07:45.956720 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:07:45.956720 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:07:45.956720 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:07:45.956720 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 21 18:07:45.956720 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:07:45.956720 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:07:45.956720 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:07:45.956720 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:07:45.956720 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:07:45.956720 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:07:45.956720 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:07:45.956720 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:07:45.958074 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:07:45.958074 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:07:45.958074 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:07:45.958074 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:07:45.958074 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:07:45.958074 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:07:45.958074 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:07:45.958074 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:07:45.958074 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:07:45.958074 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:07:45.958074 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:07:45.958074 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:07:45.958074 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:07:45.958074 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:07:45.958074 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:07:45.958074 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:07:45.959174 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:07:45.959174 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:07:45.959174 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:07:45.959174 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:07:45.959174 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:07:45.959174 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:07:45.959174 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:07:45.959174 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:07:45.959174 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:07:45.959174 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:07:45.959174 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 21 18:07:45.959174 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:07:45.959174 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 21 18:07:45.959174 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:07:45.959174 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 574/1165] 10.222.0.23 () {56 vars in 1117 bytes} [Tue Jun 21 18:07:45 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:07:45.964118 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1763859a-5330-4e91-a4d9-f5b8fd5c0cd7 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:45.984554 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:46.082023 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1763859a-5330-4e91-a4d9-f5b8fd5c0cd7 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:46.127257 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ce7316-7447-4420-8aa7-f32a53f3fdda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:46.127923 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ce7316-7447-4420-8aa7-f32a53f3fdda None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:46.128197 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ce7316-7447-4420-8aa7-f32a53f3fdda None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:46.128502 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ce7316-7447-4420-8aa7-f32a53f3fdda None admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:46.129227 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ce7316-7447-4420-8aa7-f32a53f3fdda None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:07:46.129527 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ce7316-7447-4420-8aa7-f32a53f3fdda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:46.130876 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ce7316-7447-4420-8aa7-f32a53f3fdda None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:46.140070 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 592/1166] 10.222.0.23 () {56 vars in 1115 bytes} [Tue Jun 21 18:07:45 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:07:48.236532 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956bc61-a3ea-487f-968c-dc7d116dbd9e None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:48.237076 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956bc61-a3ea-487f-968c-dc7d116dbd9e None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:48.237076 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956bc61-a3ea-487f-968c-dc7d116dbd9e None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:48.238455 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 575/1167] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:07:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:07:48.245484 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac9ac2a-71ed-47b4-8c66-55e35762f8ca None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:48.246006 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac9ac2a-71ed-47b4-8c66-55e35762f8ca None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:48.246107 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac9ac2a-71ed-47b4-8c66-55e35762f8ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:48.263371 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-dac9ac2a-71ed-47b4-8c66-55e35762f8ca None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:07:48.264617 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-dac9ac2a-71ed-47b4-8c66-55e35762f8ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:07:48.335745 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 593/1168] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:07:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:07:48.360810 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9956bc61-a3ea-487f-968c-dc7d116dbd9e None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:48.429011 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:48.472503 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9956bc61-a3ea-487f-968c-dc7d116dbd9e None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:48.513993 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c2bc3e-d421-449f-9f93-0c38a19750ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:48.514594 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c2bc3e-d421-449f-9f93-0c38a19750ad None admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:48.514884 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c2bc3e-d421-449f-9f93-0c38a19750ad None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:48.515167 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c2bc3e-d421-449f-9f93-0c38a19750ad None admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:48.515944 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c2bc3e-d421-449f-9f93-0c38a19750ad None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:48.517398 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c2bc3e-d421-449f-9f93-0c38a19750ad None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:48.590033 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 576/1169] 10.222.0.23 () {60 vars in 1138 bytes} [Tue Jun 21 18:07:48 2022] POST /identity/v3/domains => generated 267 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:07:50.712828 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d08236-6dd0-4cd3-b3d6-a46ec392a1a7 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:50.713433 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d08236-6dd0-4cd3-b3d6-a46ec392a1a7 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:50.713433 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d08236-6dd0-4cd3-b3d6-a46ec392a1a7 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:50.714767 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 594/1170] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:07:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:07:50.720693 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e09d0e-97ff-4617-8e5a-4e07d7cb5ccf None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:50.721178 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e09d0e-97ff-4617-8e5a-4e07d7cb5ccf None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:50.721290 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e09d0e-97ff-4617-8e5a-4e07d7cb5ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:50.739437 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-11e09d0e-97ff-4617-8e5a-4e07d7cb5ccf None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:07:50.740685 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-11e09d0e-97ff-4617-8e5a-4e07d7cb5ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:07:50.816711 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 577/1171] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:07:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:07:50.841888 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d08236-6dd0-4cd3-b3d6-a46ec392a1a7 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:50.858088 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:50.955397 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d08236-6dd0-4cd3-b3d6-a46ec392a1a7 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:51.011610 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e07c35-704d-43a9-9cfd-4400702ceb47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:51.013056 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e07c35-704d-43a9-9cfd-4400702ceb47 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:51.013567 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e07c35-704d-43a9-9cfd-4400702ceb47 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:51.014047 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e07c35-704d-43a9-9cfd-4400702ceb47 None admin] PATH_INFO: `/v3/domains/29bafcd58e494908a1192cd08b09ca21` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:51.035039 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e07c35-704d-43a9-9cfd-4400702ceb47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=29bafcd58e494908a1192cd08b09ca21)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:51.037814 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e07c35-704d-43a9-9cfd-4400702ceb47 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:51.041748 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 595/1172] 10.222.0.23 () {56 vars in 1183 bytes} [Tue Jun 21 18:07:50 2022] GET /identity/v3/domains/29bafcd58e494908a1192cd08b09ca21 => generated 267 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:07:51.048642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e09d0e-97ff-4617-8e5a-4e07d7cb5ccf None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:51.111192 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:51.165517 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e09d0e-97ff-4617-8e5a-4e07d7cb5ccf None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:51.202507 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4bf126-af26-412c-a5f0-7b1360351746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:51.203176 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4bf126-af26-412c-a5f0-7b1360351746 None admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:51.203458 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4bf126-af26-412c-a5f0-7b1360351746 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:51.203733 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4bf126-af26-412c-a5f0-7b1360351746 None admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:51.204479 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4bf126-af26-412c-a5f0-7b1360351746 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:51.205922 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4bf126-af26-412c-a5f0-7b1360351746 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:51.317384 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 578/1173] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Jun 21 18:07:51 2022] POST /identity/v3/projects => generated 361 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:07:53.503260 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e555f58b-9b7e-4e7a-b455-3d8a6d4e7995 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:53.503827 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e555f58b-9b7e-4e7a-b455-3d8a6d4e7995 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:53.503827 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e555f58b-9b7e-4e7a-b455-3d8a6d4e7995 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:53.505228 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 596/1174] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:07:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:07:53.511337 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50239359-cdd7-4372-b61f-137b121952e4 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:53.511861 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50239359-cdd7-4372-b61f-137b121952e4 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:53.511861 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50239359-cdd7-4372-b61f-137b121952e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:53.528622 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-50239359-cdd7-4372-b61f-137b121952e4 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:07:53.529815 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-50239359-cdd7-4372-b61f-137b121952e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:07:53.598099 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 579/1175] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:07:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:07:53.624364 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e555f58b-9b7e-4e7a-b455-3d8a6d4e7995 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:53.698039 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:53.742668 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e555f58b-9b7e-4e7a-b455-3d8a6d4e7995 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:53.781590 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27e4e13-3d32-41fa-be05-2be7d37b6054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:53.782368 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e4e13-3d32-41fa-be05-2be7d37b6054 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:53.782649 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e4e13-3d32-41fa-be05-2be7d37b6054 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:53.782925 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e4e13-3d32-41fa-be05-2be7d37b6054 None admin] PATH_INFO: `/v3/domains/29bafcd58e494908a1192cd08b09ca21` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:53.784108 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27e4e13-3d32-41fa-be05-2be7d37b6054 None admin] RBAC: Authorizing `identity:get_domain(domain_id=29bafcd58e494908a1192cd08b09ca21)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:53.785572 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27e4e13-3d32-41fa-be05-2be7d37b6054 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:53.787211 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 597/1176] 10.222.0.23 () {56 vars in 1183 bytes} [Tue Jun 21 18:07:53 2022] GET /identity/v3/domains/29bafcd58e494908a1192cd08b09ca21 => generated 267 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:07:53.793610 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50239359-cdd7-4372-b61f-137b121952e4 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:53.837170 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:53.914624 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50239359-cdd7-4372-b61f-137b121952e4 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:53.955382 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c09cb24-1061-4d43-b498-8e6636a45431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:53.956163 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09cb24-1061-4d43-b498-8e6636a45431 None admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:53.956482 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09cb24-1061-4d43-b498-8e6636a45431 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:53.956761 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09cb24-1061-4d43-b498-8e6636a45431 None admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:53.957502 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c09cb24-1061-4d43-b498-8e6636a45431 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:53.958972 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c09cb24-1061-4d43-b498-8e6636a45431 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:54.053259 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 580/1177] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Jun 21 18:07:53 2022] POST /identity/v3/users => generated 311 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:07:56.274626 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddb1693-b7d3-4ba9-bec0-8c66da50e5f5 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:56.275243 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddb1693-b7d3-4ba9-bec0-8c66da50e5f5 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:56.275243 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddb1693-b7d3-4ba9-bec0-8c66da50e5f5 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:56.278030 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 598/1178] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:07:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:07:56.284247 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce4abc2-de96-43ca-9829-2e81b7d51778 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:56.284828 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce4abc2-de96-43ca-9829-2e81b7d51778 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:56.284828 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce4abc2-de96-43ca-9829-2e81b7d51778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:56.303359 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-8ce4abc2-de96-43ca-9829-2e81b7d51778 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:07:56.304766 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-8ce4abc2-de96-43ca-9829-2e81b7d51778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:07:56.384673 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 581/1179] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:07:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:07:56.410033 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddb1693-b7d3-4ba9-bec0-8c66da50e5f5 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:56.520803 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddb1693-b7d3-4ba9-bec0-8c66da50e5f5 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:56.557910 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee6400d-e92b-493d-a0cb-29f1da40ae24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:56.558636 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee6400d-e92b-493d-a0cb-29f1da40ae24 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:56.558920 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee6400d-e92b-493d-a0cb-29f1da40ae24 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:56.559193 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee6400d-e92b-493d-a0cb-29f1da40ae24 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:56.567729 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:56.571751 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee6400d-e92b-493d-a0cb-29f1da40ae24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:07:56.579766 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-1ee6400d-e92b-493d-a0cb-29f1da40ae24 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:07:56.579766 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:07:56.579766 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:07:56.579766 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:07:56.579766 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:07:56.579766 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:07:56.579766 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:07:56.579766 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:07:56.579766 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:07:56.579766 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:07:56.579766 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:07:56.579766 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:07:56.579766 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:07:56.579766 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:07:56.579766 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:07:56.579766 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:07:56.581480 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:07:56.581480 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:07:56.581480 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:07:56.581480 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:07:56.581480 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:07:56.581480 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:07:56.581480 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:07:56.581480 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:07:56.581480 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:07:56.581480 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:07:56.581480 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:07:56.581480 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:07:56.581480 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:07:56.581480 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:07:56.581480 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:07:56.581480 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:07:56.583050 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:07:56.583050 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:07:56.583050 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:07:56.583050 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:07:56.583050 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:07:56.583050 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:07:56.583050 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:07:56.583050 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:07:56.583050 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:07:56.583050 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:07:56.583050 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:07:56.583050 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:07:56.583050 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 599/1180] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jun 21 18:07:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:07:56.588091 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce4abc2-de96-43ca-9829-2e81b7d51778 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:56.612098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:56.696871 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce4abc2-de96-43ca-9829-2e81b7d51778 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:56.732566 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a056c05-dcb6-45eb-9c44-88ea75e398ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:56.733243 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a056c05-dcb6-45eb-9c44-88ea75e398ef None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:56.733528 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a056c05-dcb6-45eb-9c44-88ea75e398ef None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:56.733813 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a056c05-dcb6-45eb-9c44-88ea75e398ef None admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:56.734563 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a056c05-dcb6-45eb-9c44-88ea75e398ef None admin] RBAC: Adding query filter params (name=admin) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:07:56.734872 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a056c05-dcb6-45eb-9c44-88ea75e398ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:56.736249 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a056c05-dcb6-45eb-9c44-88ea75e398ef None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:56.745294 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 582/1181] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:07:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:07:56.751397 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee6400d-e92b-493d-a0cb-29f1da40ae24 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:56.857222 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee6400d-e92b-493d-a0cb-29f1da40ae24 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:56.897150 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bea71ff-c4ef-42ad-b3ba-2293cce62394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:56.898069 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bea71ff-c4ef-42ad-b3ba-2293cce62394 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:56.898917 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bea71ff-c4ef-42ad-b3ba-2293cce62394 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:56.899195 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bea71ff-c4ef-42ad-b3ba-2293cce62394 None admin] PATH_INFO: `/v3/users/d1e90649986641aba03fcc79d8b027fc` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:56.915391 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bea71ff-c4ef-42ad-b3ba-2293cce62394 None admin] RBAC: Authorizing `identity:get_user(user_id=d1e90649986641aba03fcc79d8b027fc)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:56.916925 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bea71ff-c4ef-42ad-b3ba-2293cce62394 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:56.918745 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 600/1182] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jun 21 18:07:56 2022] GET /identity/v3/users/d1e90649986641aba03fcc79d8b027fc => generated 311 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:07:56.928860 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a056c05-dcb6-45eb-9c44-88ea75e398ef None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:57.036957 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:57.059472 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a056c05-dcb6-45eb-9c44-88ea75e398ef None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:57.097712 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d3bfc9-025b-4fd8-9582-c9626d20e0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:57.098428 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3bfc9-025b-4fd8-9582-c9626d20e0a7 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:57.098752 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3bfc9-025b-4fd8-9582-c9626d20e0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:57.099072 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3bfc9-025b-4fd8-9582-c9626d20e0a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:57.178770 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d3bfc9-025b-4fd8-9582-c9626d20e0a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:57.181510 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d3bfc9-025b-4fd8-9582-c9626d20e0a7 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:57.267894 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:57.287997 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 583/1183] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:07:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 18:07:57.294863 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bea71ff-c4ef-42ad-b3ba-2293cce62394 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:57.358359 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:57.403716 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bea71ff-c4ef-42ad-b3ba-2293cce62394 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:57.439900 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721cfc91-9dd6-40ff-814b-ad320c5b3d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:57.440707 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721cfc91-9dd6-40ff-814b-ad320c5b3d88 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:57.440990 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721cfc91-9dd6-40ff-814b-ad320c5b3d88 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:57.441260 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721cfc91-9dd6-40ff-814b-ad320c5b3d88 None admin] PATH_INFO: `/v3/projects/63c7020a79dc4763a2ce78833dd98b53` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:57.442472 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721cfc91-9dd6-40ff-814b-ad320c5b3d88 None admin] RBAC: Authorizing `identity:get_project(project_id=63c7020a79dc4763a2ce78833dd98b53)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:57.443911 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721cfc91-9dd6-40ff-814b-ad320c5b3d88 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:57.446430 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 601/1184] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jun 21 18:07:57 2022] GET /identity/v3/projects/63c7020a79dc4763a2ce78833dd98b53 => generated 361 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:07:57.452796 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d3bfc9-025b-4fd8-9582-c9626d20e0a7 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:57.568235 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d3bfc9-025b-4fd8-9582-c9626d20e0a7 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:57.610037 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bfc57a-2ae3-4e76-95cb-cc7d0d79ed00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:57.610713 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bfc57a-2ae3-4e76-95cb-cc7d0d79ed00 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:57.610997 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bfc57a-2ae3-4e76-95cb-cc7d0d79ed00 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:57.611272 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bfc57a-2ae3-4e76-95cb-cc7d0d79ed00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:57.612134 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bfc57a-2ae3-4e76-95cb-cc7d0d79ed00 None admin] RBAC: Adding query filter params (role.id=745ed9065a8143e384ebbd694b29635c, scope.project.id=63c7020a79dc4763a2ce78833dd98b53, user.id=d1e90649986641aba03fcc79d8b027fc) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:07:57.612475 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bfc57a-2ae3-4e76-95cb-cc7d0d79ed00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:07:57.613988 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bfc57a-2ae3-4e76-95cb-cc7d0d79ed00 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:07:57.621249 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 584/1185] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jun 21 18:07:57 2022] GET /identity/v3/role_assignments?user.id=d1e90649986641aba03fcc79d8b027fc&scope.project.id=63c7020a79dc4763a2ce78833dd98b53&role.id=745ed9065a8143e384ebbd694b29635c => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:07:59.694961 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb6850f-eef0-431e-b488-cc4420a0f14a None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:59.695601 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb6850f-eef0-431e-b488-cc4420a0f14a None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:59.695601 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb6850f-eef0-431e-b488-cc4420a0f14a None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:59.697010 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 602/1186] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:07:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:07:59.703219 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b1242e-d07b-46f2-8889-2d60a8bd05c1 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:59.703844 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b1242e-d07b-46f2-8889-2d60a8bd05c1 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:59.703844 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b1242e-d07b-46f2-8889-2d60a8bd05c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:59.722079 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-a9b1242e-d07b-46f2-8889-2d60a8bd05c1 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:07:59.723338 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-a9b1242e-d07b-46f2-8889-2d60a8bd05c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:07:59.773887 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:59.802664 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 585/1187] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:07:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:07:59.826875 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb6850f-eef0-431e-b488-cc4420a0f14a None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:07:59.944008 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb6850f-eef0-431e-b488-cc4420a0f14a None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:07:59.973207 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:07:59.983696 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0d9ad0-6718-4295-937b-affe61f5d37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:07:59.984546 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0d9ad0-6718-4295-937b-affe61f5d37d None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:07:59.984838 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0d9ad0-6718-4295-937b-affe61f5d37d None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:07:59.985116 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0d9ad0-6718-4295-937b-affe61f5d37d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:07:59.997044 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0d9ad0-6718-4295-937b-affe61f5d37d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:08:00.004845 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-5e0d9ad0-6718-4295-937b-affe61f5d37d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:08:00.004845 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:08:00.004845 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:08:00.004845 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:08:00.004845 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:08:00.004845 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:08:00.004845 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:08:00.004845 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:08:00.004845 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:08:00.004845 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:08:00.004845 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:08:00.004845 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:08:00.004845 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:08:00.004845 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:08:00.004845 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:08:00.004845 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:08:00.006704 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:08:00.006704 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:08:00.006704 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:08:00.006704 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:08:00.006704 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:08:00.006704 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:08:00.006704 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:08:00.006704 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:08:00.006704 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:08:00.006704 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:08:00.006704 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:08:00.006704 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:08:00.006704 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:08:00.006704 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:08:00.006704 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:08:00.006704 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:08:00.008814 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:08:00.008814 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:08:00.008814 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:08:00.008814 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:08:00.008814 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:08:00.008814 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:08:00.008814 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:08:00.008814 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:08:00.008814 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:08:00.008814 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:08:00.008814 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:08:00.008814 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:08:00.008814 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 603/1188] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jun 21 18:07:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:08:00.014364 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b1242e-d07b-46f2-8889-2d60a8bd05c1 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:08:00.121512 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b1242e-d07b-46f2-8889-2d60a8bd05c1 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:08:00.158178 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6793863-736b-4a03-9bce-ed6914b9d55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:08:00.158861 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6793863-736b-4a03-9bce-ed6914b9d55f None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:08:00.159152 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6793863-736b-4a03-9bce-ed6914b9d55f None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:08:00.159433 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6793863-736b-4a03-9bce-ed6914b9d55f None admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:08:00.160188 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6793863-736b-4a03-9bce-ed6914b9d55f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:08:00.160535 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6793863-736b-4a03-9bce-ed6914b9d55f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:08:00.161914 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6793863-736b-4a03-9bce-ed6914b9d55f None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:08:00.171279 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 586/1189] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:08:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:08:00.177311 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0d9ad0-6718-4295-937b-affe61f5d37d None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:08:00.286368 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0d9ad0-6718-4295-937b-affe61f5d37d None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:08:00.324519 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccae5c3-6a3e-4c47-a0d2-32b01fcf67ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:08:00.325408 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccae5c3-6a3e-4c47-a0d2-32b01fcf67ac None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:08:00.325692 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccae5c3-6a3e-4c47-a0d2-32b01fcf67ac None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:08:00.325967 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccae5c3-6a3e-4c47-a0d2-32b01fcf67ac None admin] PATH_INFO: `/v3/users/d1e90649986641aba03fcc79d8b027fc` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:08:00.327212 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccae5c3-6a3e-4c47-a0d2-32b01fcf67ac None admin] RBAC: Authorizing `identity:get_user(user_id=d1e90649986641aba03fcc79d8b027fc)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:08:00.328693 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccae5c3-6a3e-4c47-a0d2-32b01fcf67ac None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:08:00.330371 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 604/1190] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jun 21 18:08:00 2022] GET /identity/v3/users/d1e90649986641aba03fcc79d8b027fc => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:08:00.336335 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6793863-736b-4a03-9bce-ed6914b9d55f None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:08:00.378798 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:08:00.442571 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6793863-736b-4a03-9bce-ed6914b9d55f None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:08:00.480297 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d21df5-7008-45df-9090-5aaee9bd3d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:08:00.481214 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d21df5-7008-45df-9090-5aaee9bd3d9b None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:08:00.481504 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d21df5-7008-45df-9090-5aaee9bd3d9b None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:08:00.481784 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d21df5-7008-45df-9090-5aaee9bd3d9b None admin] PATH_INFO: `/v3/projects/63c7020a79dc4763a2ce78833dd98b53` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:08:00.483032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d21df5-7008-45df-9090-5aaee9bd3d9b None admin] RBAC: Authorizing `identity:get_project(project_id=63c7020a79dc4763a2ce78833dd98b53)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:08:00.484513 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d21df5-7008-45df-9090-5aaee9bd3d9b None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:08:00.486289 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 587/1191] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jun 21 18:08:00 2022] GET /identity/v3/projects/63c7020a79dc4763a2ce78833dd98b53 => generated 361 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:08:00.494491 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccae5c3-6a3e-4c47-a0d2-32b01fcf67ac None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:08:00.585297 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:08:00.634645 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccae5c3-6a3e-4c47-a0d2-32b01fcf67ac None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:08:00.670583 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbe390d-407c-4743-a0d1-663ea622c06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:08:00.671471 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe390d-407c-4743-a0d1-663ea622c06c None admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:08:00.671751 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe390d-407c-4743-a0d1-663ea622c06c None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:08:00.672035 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe390d-407c-4743-a0d1-663ea622c06c None admin] PATH_INFO: `/v3/projects/63c7020a79dc4763a2ce78833dd98b53/users/d1e90649986641aba03fcc79d8b027fc/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:08:00.674066 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbe390d-407c-4743-a0d1-663ea622c06c None admin] RBAC: Authorizing `identity:create_grant(project_id=63c7020a79dc4763a2ce78833dd98b53, user_id=d1e90649986641aba03fcc79d8b027fc, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:08:00.675495 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbe390d-407c-4743-a0d1-663ea622c06c None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:08:00.753140 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 605/1192] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jun 21 18:08:00 2022] PUT /identity/v3/projects/63c7020a79dc4763a2ce78833dd98b53/users/d1e90649986641aba03fcc79d8b027fc/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:08:02.829765 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac074cc6-69fe-4083-b7d0-f5b84b62b194 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:08:02.830311 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac074cc6-69fe-4083-b7d0-f5b84b62b194 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:08:02.830311 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac074cc6-69fe-4083-b7d0-f5b84b62b194 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:08:02.833093 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 588/1193] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:08:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:08:02.838862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48946bf9-8266-4bcc-ae46-52896a9bda5a None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:08:02.839338 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48946bf9-8266-4bcc-ae46-52896a9bda5a None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:08:02.839426 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48946bf9-8266-4bcc-ae46-52896a9bda5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:08:02.857101 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-48946bf9-8266-4bcc-ae46-52896a9bda5a None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:08:02.858295 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-48946bf9-8266-4bcc-ae46-52896a9bda5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:08:02.930976 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 606/1194] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:08:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:08:02.954868 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac074cc6-69fe-4083-b7d0-f5b84b62b194 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:08:03.063399 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac074cc6-69fe-4083-b7d0-f5b84b62b194 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:08:03.071973 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:08:03.101729 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e5482-f932-493b-9eed-038cb8670c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:08:03.102472 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e5482-f932-493b-9eed-038cb8670c48 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:08:03.102756 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e5482-f932-493b-9eed-038cb8670c48 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:08:03.103030 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e5482-f932-493b-9eed-038cb8670c48 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:08:03.113876 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5e5482-f932-493b-9eed-038cb8670c48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:08:03.122471 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-7d5e5482-f932-493b-9eed-038cb8670c48 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:08:03.122471 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:08:03.122471 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:08:03.122471 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:08:03.122471 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:08:03.122471 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:08:03.122471 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:08:03.122471 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:08:03.122471 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:08:03.122471 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:08:03.122471 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:08:03.122471 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:08:03.122471 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:08:03.122471 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:08:03.122471 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:08:03.122471 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:08:03.124438 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:08:03.124438 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:08:03.124438 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:08:03.124438 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:08:03.124438 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:08:03.124438 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:08:03.124438 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:08:03.124438 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:08:03.124438 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:08:03.124438 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:08:03.124438 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:08:03.124438 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:08:03.124438 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:08:03.124438 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:08:03.124438 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:08:03.124438 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:08:03.126504 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:08:03.126504 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:08:03.126504 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:08:03.126504 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:08:03.126504 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:08:03.126504 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:08:03.126504 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:08:03.126504 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:08:03.126504 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:08:03.126504 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:08:03.126504 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:08:03.126504 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:08:03.126504 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 589/1195] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jun 21 18:08:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:08:03.134374 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48946bf9-8266-4bcc-ae46-52896a9bda5a None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:08:03.189397 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:08:03.252757 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48946bf9-8266-4bcc-ae46-52896a9bda5a None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:08:03.289982 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51eaa7e-f26c-4b1e-be5f-bc490ce2476a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:08:03.290690 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51eaa7e-f26c-4b1e-be5f-bc490ce2476a None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:08:03.290975 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51eaa7e-f26c-4b1e-be5f-bc490ce2476a None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:08:03.291249 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51eaa7e-f26c-4b1e-be5f-bc490ce2476a None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:08:03.292035 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51eaa7e-f26c-4b1e-be5f-bc490ce2476a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:08:03.292343 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51eaa7e-f26c-4b1e-be5f-bc490ce2476a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:08:03.293841 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51eaa7e-f26c-4b1e-be5f-bc490ce2476a None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:08:03.303861 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 607/1196] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:08:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:08:03.310699 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e5482-f932-493b-9eed-038cb8670c48 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:08:03.430699 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e5482-f932-493b-9eed-038cb8670c48 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:08:03.469262 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a17a0c4-74b5-4112-a48a-2ff9e1871079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:08:03.470230 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17a0c4-74b5-4112-a48a-2ff9e1871079 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:08:03.470631 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17a0c4-74b5-4112-a48a-2ff9e1871079 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:08:03.470953 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17a0c4-74b5-4112-a48a-2ff9e1871079 None admin] PATH_INFO: `/v3/users/d1e90649986641aba03fcc79d8b027fc` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:08:03.472260 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a17a0c4-74b5-4112-a48a-2ff9e1871079 None admin] RBAC: Authorizing `identity:get_user(user_id=d1e90649986641aba03fcc79d8b027fc)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:08:03.473810 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a17a0c4-74b5-4112-a48a-2ff9e1871079 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:08:03.475819 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 590/1197] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jun 21 18:08:03 2022] GET /identity/v3/users/d1e90649986641aba03fcc79d8b027fc => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:08:03.482234 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51eaa7e-f26c-4b1e-be5f-bc490ce2476a None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:08:03.595732 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51eaa7e-f26c-4b1e-be5f-bc490ce2476a None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:08:03.604285 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:08:03.631355 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ea488a-f174-4acc-adc6-a2db4fa17aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:08:03.632055 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea488a-f174-4acc-adc6-a2db4fa17aab None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:08:03.632338 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea488a-f174-4acc-adc6-a2db4fa17aab None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:08:03.633158 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea488a-f174-4acc-adc6-a2db4fa17aab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:08:03.706457 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ea488a-f174-4acc-adc6-a2db4fa17aab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:08:03.707823 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ea488a-f174-4acc-adc6-a2db4fa17aab None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:08:03.812311 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 608/1198] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:08:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 18:08:03.822493 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a17a0c4-74b5-4112-a48a-2ff9e1871079 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:08:03.873431 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:08:03.952198 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a17a0c4-74b5-4112-a48a-2ff9e1871079 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:08:03.994507 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1a9356-6a4b-47b5-9d56-07e273945a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:08:03.995321 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a9356-6a4b-47b5-9d56-07e273945a26 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:08:03.995606 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a9356-6a4b-47b5-9d56-07e273945a26 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:08:03.995885 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a9356-6a4b-47b5-9d56-07e273945a26 None admin] PATH_INFO: `/v3/projects/63c7020a79dc4763a2ce78833dd98b53` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:08:03.997257 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1a9356-6a4b-47b5-9d56-07e273945a26 None admin] RBAC: Authorizing `identity:get_project(project_id=63c7020a79dc4763a2ce78833dd98b53)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:08:03.998775 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1a9356-6a4b-47b5-9d56-07e273945a26 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:08:04.000707 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 591/1199] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jun 21 18:08:03 2022] GET /identity/v3/projects/63c7020a79dc4763a2ce78833dd98b53 => generated 361 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:08:04.007183 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ea488a-f174-4acc-adc6-a2db4fa17aab None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:08:04.016547 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:08:04.122446 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ea488a-f174-4acc-adc6-a2db4fa17aab None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:08:04.159074 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad16d56-cb40-46fc-8456-ead3a55a2890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:08:04.159742 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad16d56-cb40-46fc-8456-ead3a55a2890 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:08:04.160024 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad16d56-cb40-46fc-8456-ead3a55a2890 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:08:04.160301 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad16d56-cb40-46fc-8456-ead3a55a2890 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:08:04.161203 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad16d56-cb40-46fc-8456-ead3a55a2890 None admin] RBAC: Adding query filter params (role.id=745ed9065a8143e384ebbd694b29635c, scope.project.id=63c7020a79dc4763a2ce78833dd98b53, user.id=d1e90649986641aba03fcc79d8b027fc) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:08:04.161514 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad16d56-cb40-46fc-8456-ead3a55a2890 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:08:04.162952 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad16d56-cb40-46fc-8456-ead3a55a2890 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:08:04.170369 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 609/1200] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jun 21 18:08:04 2022] GET /identity/v3/role_assignments?user.id=d1e90649986641aba03fcc79d8b027fc&scope.project.id=63c7020a79dc4763a2ce78833dd98b53&role.id=745ed9065a8143e384ebbd694b29635c => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:08:13.989707 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=49 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:08:14.167324 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=65 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:21.236149 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767818ed-fb0c-4827-8322-0549aaae4ce1 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:21.236796 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767818ed-fb0c-4827-8322-0549aaae4ce1 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:21.236796 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767818ed-fb0c-4827-8322-0549aaae4ce1 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:21.238124 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 592/1201] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:09:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:09:21.243881 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d44fce4-3523-4337-a71e-c2b6144790e2 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:21.244321 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d44fce4-3523-4337-a71e-c2b6144790e2 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:21.244467 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d44fce4-3523-4337-a71e-c2b6144790e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:21.261901 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-0d44fce4-3523-4337-a71e-c2b6144790e2 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:09:21.263082 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-0d44fce4-3523-4337-a71e-c2b6144790e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:09:21.331601 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 610/1202] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:09:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:09:21.357021 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767818ed-fb0c-4827-8322-0549aaae4ce1 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:21.466874 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767818ed-fb0c-4827-8322-0549aaae4ce1 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:21.501572 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50012019-4ecf-4b31-8ddc-545ea4a00424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:21.502224 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50012019-4ecf-4b31-8ddc-545ea4a00424 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:21.502505 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50012019-4ecf-4b31-8ddc-545ea4a00424 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:21.502774 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50012019-4ecf-4b31-8ddc-545ea4a00424 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:21.513991 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50012019-4ecf-4b31-8ddc-545ea4a00424 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:21.515430 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50012019-4ecf-4b31-8ddc-545ea4a00424 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:21.533778 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-50012019-4ecf-4b31-8ddc-545ea4a00424 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:09:21.533778 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:09:21.533778 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:09:21.533778 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:09:21.533778 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:09:21.533778 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:09:21.533778 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:09:21.533778 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:09:21.533778 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:09:21.533778 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:09:21.533778 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:09:21.533778 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:09:21.533778 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:09:21.533778 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:09:21.533778 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:09:21.535102 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:09:21.535102 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:09:21.535102 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:09:21.535102 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:09:21.535102 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:09:21.535102 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:09:21.535102 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:09:21.535102 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:09:21.535102 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:09:21.535102 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:09:21.535102 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:09:21.535102 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:09:21.535102 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:09:21.535102 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:09:21.535102 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:09:21.535102 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:09:21.535102 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:09:21.540302 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:09:21.540302 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:09:21.540302 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:09:21.540302 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:09:21.540302 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 593/1203] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:09:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:09:21.541678 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d44fce4-3523-4337-a71e-c2b6144790e2 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:21.644324 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d44fce4-3523-4337-a71e-c2b6144790e2 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:21.672810 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:21.678620 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b92ac6-59c6-42e3-a78c-e4790a66ae9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:21.679279 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b92ac6-59c6-42e3-a78c-e4790a66ae9f None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:21.679559 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b92ac6-59c6-42e3-a78c-e4790a66ae9f None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:21.679833 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b92ac6-59c6-42e3-a78c-e4790a66ae9f None admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:21.680583 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b92ac6-59c6-42e3-a78c-e4790a66ae9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:09:21.680889 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b92ac6-59c6-42e3-a78c-e4790a66ae9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:21.682252 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b92ac6-59c6-42e3-a78c-e4790a66ae9f None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:21.693458 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 611/1204] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:09:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:09:21.699815 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50012019-4ecf-4b31-8ddc-545ea4a00424 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:21.754554 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:21.802345 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50012019-4ecf-4b31-8ddc-545ea4a00424 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:21.836668 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87130274-7936-4519-9672-66111e0fe985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:21.837342 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87130274-7936-4519-9672-66111e0fe985 None admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:21.837630 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87130274-7936-4519-9672-66111e0fe985 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:21.837937 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87130274-7936-4519-9672-66111e0fe985 None admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:21.838665 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87130274-7936-4519-9672-66111e0fe985 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:21.840092 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87130274-7936-4519-9672-66111e0fe985 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:21.945290 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 594/1205] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Jun 21 18:09:21 2022] POST /identity/v3/users => generated 251 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:09:24.030596 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a9f459-c375-471c-a341-f41b1db925ea None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:24.031190 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a9f459-c375-471c-a341-f41b1db925ea None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:24.031190 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a9f459-c375-471c-a341-f41b1db925ea None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:24.032556 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 612/1206] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:09:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:09:24.038012 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017de70e-352b-4107-8943-cbc5a3dc35e4 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:24.038701 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017de70e-352b-4107-8943-cbc5a3dc35e4 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:24.038701 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017de70e-352b-4107-8943-cbc5a3dc35e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:24.055508 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-017de70e-352b-4107-8943-cbc5a3dc35e4 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:09:24.056742 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-017de70e-352b-4107-8943-cbc5a3dc35e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:09:24.123349 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 595/1207] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:09:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:09:24.147921 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a9f459-c375-471c-a341-f41b1db925ea None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:24.248815 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a9f459-c375-471c-a341-f41b1db925ea None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:24.281397 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4411eda2-f806-458d-9ec8-f74cdc3e9c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:24.282590 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411eda2-f806-458d-9ec8-f74cdc3e9c22 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:24.282590 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411eda2-f806-458d-9ec8-f74cdc3e9c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:24.282780 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411eda2-f806-458d-9ec8-f74cdc3e9c22 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:24.293801 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4411eda2-f806-458d-9ec8-f74cdc3e9c22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:09:24.301436 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-4411eda2-f806-458d-9ec8-f74cdc3e9c22 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:09:24.301436 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:09:24.301436 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:09:24.301436 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:09:24.301436 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:09:24.301436 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:09:24.301436 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:09:24.301436 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:09:24.301436 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:09:24.301436 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:09:24.301436 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:09:24.301436 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:09:24.301436 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:09:24.301436 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:09:24.301436 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:09:24.301436 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:09:24.302473 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:09:24.302473 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:09:24.302473 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:09:24.302473 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:09:24.302473 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:09:24.302473 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:09:24.302473 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:09:24.302473 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:09:24.302473 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:09:24.302473 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:09:24.302473 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:09:24.302473 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:09:24.302473 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:09:24.302473 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:09:24.302473 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:09:24.302473 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:09:24.304087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:09:24.304087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:09:24.304087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:09:24.304087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:09:24.304087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:09:24.304087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:09:24.304087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:09:24.304087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:09:24.304087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:09:24.304087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:09:24.304087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:09:24.304087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:09:24.304087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 613/1208] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jun 21 18:09:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:09:24.309388 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017de70e-352b-4107-8943-cbc5a3dc35e4 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:24.348787 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:24.408492 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017de70e-352b-4107-8943-cbc5a3dc35e4 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:24.443211 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d3bfa-d7a1-4169-b4b4-9f0ccd7eaf98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:24.444308 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d3bfa-d7a1-4169-b4b4-9f0ccd7eaf98 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:24.444715 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d3bfa-d7a1-4169-b4b4-9f0ccd7eaf98 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:24.445011 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d3bfa-d7a1-4169-b4b4-9f0ccd7eaf98 None admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:24.445794 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64d3bfa-d7a1-4169-b4b4-9f0ccd7eaf98 None admin] RBAC: Adding query filter params (name=service) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:09:24.446110 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64d3bfa-d7a1-4169-b4b4-9f0ccd7eaf98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:24.447490 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64d3bfa-d7a1-4169-b4b4-9f0ccd7eaf98 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:24.455795 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 596/1209] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jun 21 18:09:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:09:24.462668 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4411eda2-f806-458d-9ec8-f74cdc3e9c22 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:24.515332 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:24.567313 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4411eda2-f806-458d-9ec8-f74cdc3e9c22 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:24.602793 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf824fa8-d6c9-4334-b48e-c0b53483516d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:24.603590 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf824fa8-d6c9-4334-b48e-c0b53483516d None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:24.603869 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf824fa8-d6c9-4334-b48e-c0b53483516d None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:24.604142 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf824fa8-d6c9-4334-b48e-c0b53483516d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:24.614389 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf824fa8-d6c9-4334-b48e-c0b53483516d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:24.617163 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf824fa8-d6c9-4334-b48e-c0b53483516d None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:24.635489 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-cf824fa8-d6c9-4334-b48e-c0b53483516d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:09:24.635489 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:09:24.635489 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:09:24.635489 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:09:24.635489 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:09:24.635489 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:09:24.635489 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:09:24.635489 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:09:24.635489 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:09:24.635489 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:09:24.635489 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:09:24.635489 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:09:24.635489 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:09:24.635489 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:09:24.635489 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:09:24.636686 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:09:24.636686 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:09:24.636686 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:09:24.636686 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:09:24.636686 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:09:24.636686 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:09:24.636686 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:09:24.636686 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:09:24.636686 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:09:24.636686 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:09:24.636686 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:09:24.636686 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:09:24.636686 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:09:24.636686 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:09:24.636686 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:09:24.636686 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:09:24.636686 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:09:24.637863 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:09:24.637863 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:09:24.637863 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:09:24.637863 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:09:24.637863 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 614/1210] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:09:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:09:24.642731 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d3bfa-d7a1-4169-b4b4-9f0ccd7eaf98 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:24.758190 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:24.761091 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d3bfa-d7a1-4169-b4b4-9f0ccd7eaf98 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:24.794586 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486cfee2-8425-447e-8e84-e962c1684a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:24.795255 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486cfee2-8425-447e-8e84-e962c1684a3f None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:24.795537 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486cfee2-8425-447e-8e84-e962c1684a3f None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:24.795814 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486cfee2-8425-447e-8e84-e962c1684a3f None admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:24.796570 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486cfee2-8425-447e-8e84-e962c1684a3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:09:24.796877 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486cfee2-8425-447e-8e84-e962c1684a3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:24.798242 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486cfee2-8425-447e-8e84-e962c1684a3f None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:24.810500 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 597/1211] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:09:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:09:24.816262 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf824fa8-d6c9-4334-b48e-c0b53483516d None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:24.922257 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf824fa8-d6c9-4334-b48e-c0b53483516d None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:24.923837 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:24.958331 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d757696b-fbfc-4bb0-ba39-6e8a2e0075c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:24.959164 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d757696b-fbfc-4bb0-ba39-6e8a2e0075c3 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:24.959499 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d757696b-fbfc-4bb0-ba39-6e8a2e0075c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:24.959771 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d757696b-fbfc-4bb0-ba39-6e8a2e0075c3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:24.967506 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d757696b-fbfc-4bb0-ba39-6e8a2e0075c3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:24.969016 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d757696b-fbfc-4bb0-ba39-6e8a2e0075c3 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:24.983745 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-d757696b-fbfc-4bb0-ba39-6e8a2e0075c3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 21 18:09:24.983745 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:09:24.983745 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:09:24.983745 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:09:24.983745 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:09:24.983745 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:09:24.983745 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:09:24.983745 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:09:24.983745 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:09:24.983745 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:09:24.983745 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:09:24.983745 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:09:24.983745 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:09:24.983745 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:09:24.983745 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:09:24.983745 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:09:24.985011 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:09:24.985011 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:09:24.985011 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:09:24.985011 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:09:24.985011 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:09:24.985011 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:09:24.985011 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:09:24.985011 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:09:24.985011 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:09:24.985011 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:09:24.985011 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:09:24.985011 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:09:24.985011 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:09:24.985011 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:09:24.985011 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:09:24.985011 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:09:24.985011 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:09:24.985011 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:09:24.986198 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:09:24.986198 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:09:24.986198 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:09:24.986198 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:09:24.986198 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:09:24.986198 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:09:24.986198 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:09:24.986198 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:09:24.986198 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 21 18:09:24.986198 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:09:24.986198 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 615/1212] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:09:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:09:24.992283 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486cfee2-8425-447e-8e84-e962c1684a3f None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:25.095337 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486cfee2-8425-447e-8e84-e962c1684a3f None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:25.130827 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa79f37a-e6df-4cb4-9a83-95a89c933c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:25.131507 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79f37a-e6df-4cb4-9a83-95a89c933c14 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:25.131790 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79f37a-e6df-4cb4-9a83-95a89c933c14 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:25.132077 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79f37a-e6df-4cb4-9a83-95a89c933c14 None admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:25.132998 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa79f37a-e6df-4cb4-9a83-95a89c933c14 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:09:25.133311 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa79f37a-e6df-4cb4-9a83-95a89c933c14 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:25.134836 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa79f37a-e6df-4cb4-9a83-95a89c933c14 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:25.152520 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 598/1213] 10.222.0.23 () {56 vars in 1143 bytes} [Tue Jun 21 18:09:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:09:25.158203 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d757696b-fbfc-4bb0-ba39-6e8a2e0075c3 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:25.261520 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d757696b-fbfc-4bb0-ba39-6e8a2e0075c3 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:25.316564 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f8252c-7916-4410-b659-2c00c41bd737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:25.317688 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8252c-7916-4410-b659-2c00c41bd737 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:25.318190 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8252c-7916-4410-b659-2c00c41bd737 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:25.318598 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8252c-7916-4410-b659-2c00c41bd737 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:25.358081 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:25.395401 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f8252c-7916-4410-b659-2c00c41bd737 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:25.397410 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f8252c-7916-4410-b659-2c00c41bd737 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:25.499149 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 616/1214] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:09:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 18:09:25.505077 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa79f37a-e6df-4cb4-9a83-95a89c933c14 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:25.524917 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:25.606008 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa79f37a-e6df-4cb4-9a83-95a89c933c14 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:25.642112 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0059122-4582-448b-aed3-efc96bff160d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:25.642832 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0059122-4582-448b-aed3-efc96bff160d None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:25.643114 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0059122-4582-448b-aed3-efc96bff160d None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:25.643386 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0059122-4582-448b-aed3-efc96bff160d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:25.653754 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0059122-4582-448b-aed3-efc96bff160d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:25.655186 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0059122-4582-448b-aed3-efc96bff160d None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:25.672756 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-d0059122-4582-448b-aed3-efc96bff160d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:09:25.672756 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:09:25.672756 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:09:25.672756 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:09:25.672756 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:09:25.672756 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:09:25.672756 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:09:25.672756 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:09:25.672756 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:09:25.672756 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:09:25.672756 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:09:25.672756 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:09:25.672756 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:09:25.672756 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:09:25.672756 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:09:25.679734 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:09:25.679734 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:09:25.679734 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:09:25.679734 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:09:25.679734 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:09:25.679734 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:09:25.679734 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:09:25.679734 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:09:25.679734 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:09:25.679734 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:09:25.679734 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:09:25.679734 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:09:25.679734 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:09:25.679734 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:09:25.679734 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:09:25.679734 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:09:25.679734 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:09:25.680709 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:09:25.680709 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:09:25.680709 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:09:25.680709 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:09:25.680709 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 599/1215] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:09:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:09:25.681064 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f8252c-7916-4410-b659-2c00c41bd737 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:25.741590 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:25.781573 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f8252c-7916-4410-b659-2c00c41bd737 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:25.820235 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b20c671-889f-4400-826e-9bba38c7c014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:25.821076 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b20c671-889f-4400-826e-9bba38c7c014 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:25.821212 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b20c671-889f-4400-826e-9bba38c7c014 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:25.821485 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b20c671-889f-4400-826e-9bba38c7c014 None admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:25.822227 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b20c671-889f-4400-826e-9bba38c7c014 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:09:25.822534 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b20c671-889f-4400-826e-9bba38c7c014 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:25.823917 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b20c671-889f-4400-826e-9bba38c7c014 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:25.836832 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 617/1216] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:09:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:09:25.842959 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0059122-4582-448b-aed3-efc96bff160d None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:25.920375 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:25.951860 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0059122-4582-448b-aed3-efc96bff160d None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:25.985466 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a9a856-72e0-46a8-b71a-ba94ad9e8e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:25.986259 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9a856-72e0-46a8-b71a-ba94ad9e8e56 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:25.986532 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9a856-72e0-46a8-b71a-ba94ad9e8e56 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:25.986828 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9a856-72e0-46a8-b71a-ba94ad9e8e56 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:25.994338 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a9a856-72e0-46a8-b71a-ba94ad9e8e56 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:25.995854 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a9a856-72e0-46a8-b71a-ba94ad9e8e56 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:26.010309 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-c6a9a856-72e0-46a8-b71a-ba94ad9e8e56 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:09:26.010309 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:09:26.010309 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:09:26.010309 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:09:26.010309 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:09:26.010309 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:09:26.010309 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:09:26.010309 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:09:26.010309 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:09:26.010309 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:09:26.010309 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:09:26.010309 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:09:26.010309 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:09:26.010309 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:09:26.010309 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:09:26.011451 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:09:26.011451 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:09:26.011451 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:09:26.011451 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:09:26.011451 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:09:26.011451 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:09:26.011451 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:09:26.011451 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:09:26.011451 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:09:26.011451 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:09:26.011451 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:09:26.011451 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:09:26.011451 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:09:26.011451 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:09:26.011451 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:09:26.011451 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:09:26.011451 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:09:26.016890 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:09:26.016890 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:09:26.016890 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:09:26.016890 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:09:26.016890 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:09:26.016890 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:09:26.016890 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:09:26.016890 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:09:26.016890 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 600/1217] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jun 21 18:09:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:09:26.017739 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b20c671-889f-4400-826e-9bba38c7c014 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:26.122394 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b20c671-889f-4400-826e-9bba38c7c014 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:26.144668 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:26.156573 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa2af32-c21c-4c7a-8abf-3aa8d017f24d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:26.157239 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa2af32-c21c-4c7a-8abf-3aa8d017f24d None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:26.157523 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa2af32-c21c-4c7a-8abf-3aa8d017f24d None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:26.157800 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa2af32-c21c-4c7a-8abf-3aa8d017f24d None admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:26.158594 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa2af32-c21c-4c7a-8abf-3aa8d017f24d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:09:26.159127 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa2af32-c21c-4c7a-8abf-3aa8d017f24d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:26.160311 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa2af32-c21c-4c7a-8abf-3aa8d017f24d None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:26.173009 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 618/1218] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jun 21 18:09:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:09:26.179381 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a9a856-72e0-46a8-b71a-ba94ad9e8e56 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:26.278711 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a9a856-72e0-46a8-b71a-ba94ad9e8e56 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:26.309505 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:26.315312 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f8c47-9e19-45e7-b076-1d1ee764d134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:26.315972 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f8c47-9e19-45e7-b076-1d1ee764d134 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:26.316255 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f8c47-9e19-45e7-b076-1d1ee764d134 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:26.316561 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f8c47-9e19-45e7-b076-1d1ee764d134 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:26.317385 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59f8c47-9e19-45e7-b076-1d1ee764d134 None admin] RBAC: Adding query filter params (role.id=acf4e26e0ceb41ddad002dd4edf6373c, scope.project.id=2a4699d1ec70456da5f8e0aa4b1439b7, user.id=938187967efe49be9c5389d1feef81ed) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:09:26.317696 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59f8c47-9e19-45e7-b076-1d1ee764d134 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:26.319343 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59f8c47-9e19-45e7-b076-1d1ee764d134 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:26.324901 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 601/1219] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jun 21 18:09:26 2022] GET /identity/v3/role_assignments?user.id=938187967efe49be9c5389d1feef81ed&scope.project.id=2a4699d1ec70456da5f8e0aa4b1439b7&role.id=acf4e26e0ceb41ddad002dd4edf6373c => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:09:28.363647 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b4376b-ff5a-4e83-85c5-e4a8f2f5264e None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:28.364211 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b4376b-ff5a-4e83-85c5-e4a8f2f5264e None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:28.364211 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b4376b-ff5a-4e83-85c5-e4a8f2f5264e None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:28.365606 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 619/1220] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:09:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:09:28.372768 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a9d68b-8634-4269-b194-70cb19c3447a None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:28.373249 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a9d68b-8634-4269-b194-70cb19c3447a None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:28.373345 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a9d68b-8634-4269-b194-70cb19c3447a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:28.391256 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-83a9d68b-8634-4269-b194-70cb19c3447a None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:09:28.393588 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-83a9d68b-8634-4269-b194-70cb19c3447a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:09:28.462479 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 602/1221] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:09:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:09:28.486130 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b4376b-ff5a-4e83-85c5-e4a8f2f5264e None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:28.590778 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b4376b-ff5a-4e83-85c5-e4a8f2f5264e None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:28.627070 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d8e0dc-b334-45da-866d-610fd8f9c0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:28.627804 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d8e0dc-b334-45da-866d-610fd8f9c0d9 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:28.628092 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d8e0dc-b334-45da-866d-610fd8f9c0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:28.628373 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d8e0dc-b334-45da-866d-610fd8f9c0d9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:28.640826 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d8e0dc-b334-45da-866d-610fd8f9c0d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:09:28.649602 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-f2d8e0dc-b334-45da-866d-610fd8f9c0d9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:09:28.649602 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:09:28.649602 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:09:28.649602 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:09:28.649602 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:09:28.649602 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:09:28.649602 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:09:28.649602 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:09:28.649602 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:09:28.649602 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:09:28.649602 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:09:28.649602 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:09:28.649602 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:09:28.649602 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:09:28.649602 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:09:28.649602 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:09:28.650862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:09:28.650862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:09:28.650862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:09:28.650862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:09:28.650862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:09:28.650862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:09:28.650862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:09:28.650862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:09:28.650862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:09:28.650862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:09:28.650862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:09:28.650862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:09:28.650862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:09:28.650862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:09:28.650862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:09:28.650862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:09:28.652093 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:09:28.652093 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:09:28.652093 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:09:28.652093 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:09:28.652093 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:09:28.652093 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:09:28.652093 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:09:28.652093 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:09:28.652093 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:09:28.652093 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:09:28.652093 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:09:28.652093 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:09:28.652093 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 620/1222] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jun 21 18:09:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:09:28.657228 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a9d68b-8634-4269-b194-70cb19c3447a None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:28.767283 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a9d68b-8634-4269-b194-70cb19c3447a None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:28.792527 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:28.803637 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a84f19-bd0e-4806-a9d6-bf4605fb8fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:28.804435 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a84f19-bd0e-4806-a9d6-bf4605fb8fe5 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:28.808351 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a84f19-bd0e-4806-a9d6-bf4605fb8fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:28.808351 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a84f19-bd0e-4806-a9d6-bf4605fb8fe5 None admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:28.808351 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a84f19-bd0e-4806-a9d6-bf4605fb8fe5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:09:28.808351 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a84f19-bd0e-4806-a9d6-bf4605fb8fe5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:28.808351 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a84f19-bd0e-4806-a9d6-bf4605fb8fe5 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:28.815670 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 603/1223] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jun 21 18:09:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:09:28.822559 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d8e0dc-b334-45da-866d-610fd8f9c0d9 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:28.867105 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:28.925353 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d8e0dc-b334-45da-866d-610fd8f9c0d9 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:28.958518 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd8d673-e891-4e41-b04c-0685ad046390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:28.959410 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd8d673-e891-4e41-b04c-0685ad046390 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:28.959696 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd8d673-e891-4e41-b04c-0685ad046390 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:28.959976 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd8d673-e891-4e41-b04c-0685ad046390 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:28.970558 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd8d673-e891-4e41-b04c-0685ad046390 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:28.972012 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd8d673-e891-4e41-b04c-0685ad046390 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:28.990272 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-7cd8d673-e891-4e41-b04c-0685ad046390 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:09:28.990272 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:09:28.990272 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:09:28.990272 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:09:28.990272 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:09:28.990272 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:09:28.990272 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:09:28.990272 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:09:28.990272 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:09:28.990272 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:09:28.990272 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:09:28.990272 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:09:28.990272 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:09:28.990272 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:09:28.990272 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:09:28.991897 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:09:28.991897 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:09:28.991897 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:09:28.991897 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:09:28.991897 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:09:28.991897 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:09:28.991897 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:09:28.991897 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:09:28.991897 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:09:28.991897 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:09:28.991897 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:09:28.991897 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:09:28.991897 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:09:28.991897 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:09:28.991897 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:09:28.991897 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:09:28.991897 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:09:28.992944 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:09:28.992944 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:09:28.992944 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:09:28.992944 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:09:28.993550 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 621/1224] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:09:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:09:28.999431 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a84f19-bd0e-4806-a9d6-bf4605fb8fe5 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:29.103685 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a84f19-bd0e-4806-a9d6-bf4605fb8fe5 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:29.138898 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c91a8e-227f-4b37-9729-667000ad64d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:29.139484 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c91a8e-227f-4b37-9729-667000ad64d9 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:29.139769 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c91a8e-227f-4b37-9729-667000ad64d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:29.140051 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c91a8e-227f-4b37-9729-667000ad64d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:29.140891 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c91a8e-227f-4b37-9729-667000ad64d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:09:29.141272 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c91a8e-227f-4b37-9729-667000ad64d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:29.142639 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c91a8e-227f-4b37-9729-667000ad64d9 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:29.155014 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 604/1225] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:09:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:09:29.160809 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd8d673-e891-4e41-b04c-0685ad046390 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:29.253657 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:29.260921 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd8d673-e891-4e41-b04c-0685ad046390 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:29.293129 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1008866a-18e4-4afd-aa97-2e0671b14c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:29.293934 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1008866a-18e4-4afd-aa97-2e0671b14c77 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:29.294222 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1008866a-18e4-4afd-aa97-2e0671b14c77 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:29.294503 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1008866a-18e4-4afd-aa97-2e0671b14c77 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:29.302220 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1008866a-18e4-4afd-aa97-2e0671b14c77 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:29.304249 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1008866a-18e4-4afd-aa97-2e0671b14c77 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:29.319453 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-1008866a-18e4-4afd-aa97-2e0671b14c77 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 21 18:09:29.319453 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:09:29.319453 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:09:29.319453 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:09:29.319453 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:09:29.319453 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:09:29.319453 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:09:29.319453 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:09:29.319453 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:09:29.319453 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:09:29.319453 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:09:29.319453 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:09:29.319453 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:09:29.319453 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:09:29.319453 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:09:29.319453 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:09:29.320876 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:09:29.320876 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:09:29.320876 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:09:29.320876 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:09:29.320876 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:09:29.320876 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:09:29.320876 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:09:29.320876 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:09:29.320876 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:09:29.320876 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:09:29.320876 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:09:29.320876 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:09:29.320876 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:09:29.320876 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:09:29.320876 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:09:29.320876 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:09:29.320876 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:09:29.320876 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:09:29.322128 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:09:29.322128 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:09:29.322128 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:09:29.322128 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:09:29.322128 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:09:29.322128 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:09:29.322128 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:09:29.322128 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:09:29.322128 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 21 18:09:29.322128 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:09:29.322128 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 622/1226] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:09:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:09:29.327537 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c91a8e-227f-4b37-9729-667000ad64d9 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:29.374470 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:29.426540 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c91a8e-227f-4b37-9729-667000ad64d9 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:29.459023 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d787e52-12b0-4524-a524-7f7e11be69ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:29.459915 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d787e52-12b0-4524-a524-7f7e11be69ad None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:29.460203 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d787e52-12b0-4524-a524-7f7e11be69ad None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:29.460506 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d787e52-12b0-4524-a524-7f7e11be69ad None admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:29.461339 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d787e52-12b0-4524-a524-7f7e11be69ad None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:09:29.461647 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d787e52-12b0-4524-a524-7f7e11be69ad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:29.463065 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d787e52-12b0-4524-a524-7f7e11be69ad None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:29.480596 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 605/1227] 10.222.0.23 () {56 vars in 1143 bytes} [Tue Jun 21 18:09:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:09:29.486159 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1008866a-18e4-4afd-aa97-2e0671b14c77 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:29.587072 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1008866a-18e4-4afd-aa97-2e0671b14c77 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:29.620982 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e45bd6-f0a6-4c11-abd2-8815f4f6e4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:29.621783 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e45bd6-f0a6-4c11-abd2-8815f4f6e4b0 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:29.622062 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e45bd6-f0a6-4c11-abd2-8815f4f6e4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:29.622337 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e45bd6-f0a6-4c11-abd2-8815f4f6e4b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:29.632882 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e45bd6-f0a6-4c11-abd2-8815f4f6e4b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:29.635867 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e45bd6-f0a6-4c11-abd2-8815f4f6e4b0 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:29.650872 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-d3e45bd6-f0a6-4c11-abd2-8815f4f6e4b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:09:29.650872 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:09:29.650872 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:09:29.650872 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:09:29.650872 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:09:29.650872 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:09:29.650872 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:09:29.650872 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:09:29.650872 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:09:29.650872 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:09:29.650872 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:09:29.650872 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:09:29.650872 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:09:29.650872 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:09:29.650872 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:09:29.651966 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:09:29.651966 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:09:29.651966 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:09:29.651966 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:09:29.651966 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:09:29.651966 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:09:29.651966 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:09:29.651966 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:09:29.651966 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:09:29.651966 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:09:29.651966 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:09:29.651966 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:09:29.651966 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:09:29.651966 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:09:29.651966 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:09:29.651966 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:09:29.651966 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:09:29.653148 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:09:29.653148 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:09:29.653148 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:09:29.653148 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:09:29.653148 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 623/1228] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:09:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:09:29.657767 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d787e52-12b0-4524-a524-7f7e11be69ad None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:29.759442 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d787e52-12b0-4524-a524-7f7e11be69ad None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:29.761705 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:29.792824 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3c76ee-c2e8-4984-9915-100ad4c39365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:29.793488 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c76ee-c2e8-4984-9915-100ad4c39365 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:29.793770 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c76ee-c2e8-4984-9915-100ad4c39365 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:29.794043 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c76ee-c2e8-4984-9915-100ad4c39365 None admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:29.794750 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3c76ee-c2e8-4984-9915-100ad4c39365 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:09:29.795052 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3c76ee-c2e8-4984-9915-100ad4c39365 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:29.796436 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3c76ee-c2e8-4984-9915-100ad4c39365 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:29.808533 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 606/1229] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:09:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:09:29.814460 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e45bd6-f0a6-4c11-abd2-8815f4f6e4b0 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:29.824215 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:29.915395 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e45bd6-f0a6-4c11-abd2-8815f4f6e4b0 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:29.949164 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37c3144-3100-43ed-940a-bf46e725d065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:29.949987 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c3144-3100-43ed-940a-bf46e725d065 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:29.950267 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c3144-3100-43ed-940a-bf46e725d065 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:29.950537 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c3144-3100-43ed-940a-bf46e725d065 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:29.957274 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37c3144-3100-43ed-940a-bf46e725d065 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:29.961165 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37c3144-3100-43ed-940a-bf46e725d065 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:29.974013 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-d37c3144-3100-43ed-940a-bf46e725d065 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:09:29.974013 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:09:29.974013 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:09:29.974013 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:09:29.974013 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:09:29.974013 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:09:29.974013 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:09:29.974013 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:09:29.974013 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:09:29.974013 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:09:29.974013 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:09:29.974013 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:09:29.974013 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:09:29.974013 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:09:29.974013 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:09:29.975712 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:09:29.975712 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:09:29.975712 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:09:29.975712 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:09:29.975712 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:09:29.975712 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:09:29.975712 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:09:29.975712 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:09:29.975712 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:09:29.975712 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:09:29.975712 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:09:29.975712 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:09:29.975712 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:09:29.975712 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:09:29.975712 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:09:29.975712 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:09:29.975712 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:09:29.976958 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:09:29.976958 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:09:29.976958 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:09:29.976958 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:09:29.976958 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:09:29.976958 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:09:29.976958 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:09:29.976958 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:09:29.976958 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 624/1230] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jun 21 18:09:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:09:29.982420 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3c76ee-c2e8-4984-9915-100ad4c39365 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:30.081423 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3c76ee-c2e8-4984-9915-100ad4c39365 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:30.115664 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd4bf2e-324a-4087-af43-c474f071737e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:30.116421 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4bf2e-324a-4087-af43-c474f071737e None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:30.128786 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4bf2e-324a-4087-af43-c474f071737e None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:30.128786 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4bf2e-324a-4087-af43-c474f071737e None admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:30.128786 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd4bf2e-324a-4087-af43-c474f071737e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:09:30.128786 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd4bf2e-324a-4087-af43-c474f071737e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:30.128786 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd4bf2e-324a-4087-af43-c474f071737e None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:30.131433 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 607/1231] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jun 21 18:09:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:09:30.137731 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37c3144-3100-43ed-940a-bf46e725d065 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:30.204202 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:30.238664 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37c3144-3100-43ed-940a-bf46e725d065 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:30.273801 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a8d658-38a4-414e-8330-579eca460bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:30.274687 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a8d658-38a4-414e-8330-579eca460bb0 None admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:30.274967 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a8d658-38a4-414e-8330-579eca460bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:30.275239 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a8d658-38a4-414e-8330-579eca460bb0 None admin] PATH_INFO: `/v3/projects/2a4699d1ec70456da5f8e0aa4b1439b7/users/938187967efe49be9c5389d1feef81ed/roles/acf4e26e0ceb41ddad002dd4edf6373c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:30.289863 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a8d658-38a4-414e-8330-579eca460bb0 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a4699d1ec70456da5f8e0aa4b1439b7, user_id=938187967efe49be9c5389d1feef81ed, role_id=acf4e26e0ceb41ddad002dd4edf6373c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:30.291304 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a8d658-38a4-414e-8330-579eca460bb0 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:30.360152 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 625/1232] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jun 21 18:09:30 2022] PUT /identity/v3/projects/2a4699d1ec70456da5f8e0aa4b1439b7/users/938187967efe49be9c5389d1feef81ed/roles/acf4e26e0ceb41ddad002dd4edf6373c => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:09:32.412672 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e846e-76e3-424d-9704-9783ab320f3a None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:32.413218 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e846e-76e3-424d-9704-9783ab320f3a None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:32.413405 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e846e-76e3-424d-9704-9783ab320f3a None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:32.414794 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 608/1233] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:09:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:09:32.420157 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ebe890-788f-43c5-84dd-63c590868549 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:32.420597 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ebe890-788f-43c5-84dd-63c590868549 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:32.420759 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ebe890-788f-43c5-84dd-63c590868549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:32.437779 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-b8ebe890-788f-43c5-84dd-63c590868549 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:09:32.438963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-b8ebe890-788f-43c5-84dd-63c590868549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:09:32.504956 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 626/1234] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:09:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:09:32.529606 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858e846e-76e3-424d-9704-9783ab320f3a None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:32.555479 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:32.630352 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858e846e-76e3-424d-9704-9783ab320f3a None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:32.664212 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81effb71-78c3-4940-abcb-625dc3889a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:32.664967 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81effb71-78c3-4940-abcb-625dc3889a7d None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:32.665237 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81effb71-78c3-4940-abcb-625dc3889a7d None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:32.665514 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81effb71-78c3-4940-abcb-625dc3889a7d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:32.675543 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81effb71-78c3-4940-abcb-625dc3889a7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:09:32.683983 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-81effb71-78c3-4940-abcb-625dc3889a7d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:09:32.683983 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:09:32.683983 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:09:32.683983 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:09:32.683983 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:09:32.683983 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:09:32.683983 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:09:32.683983 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:09:32.683983 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:09:32.683983 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:09:32.683983 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:09:32.683983 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:09:32.683983 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:09:32.683983 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:09:32.683983 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:09:32.683983 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:09:32.687529 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:09:32.687529 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:09:32.687529 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:09:32.687529 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:09:32.687529 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:09:32.687529 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:09:32.687529 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:09:32.687529 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:09:32.687529 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:09:32.687529 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:09:32.687529 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:09:32.687529 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:09:32.687529 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:09:32.687529 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:09:32.687529 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:09:32.687529 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:09:32.688633 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:09:32.688633 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:09:32.688633 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:09:32.688633 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:09:32.688633 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:09:32.688633 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:09:32.688633 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:09:32.688633 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:09:32.688633 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:09:32.688633 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:09:32.688633 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:09:32.688633 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:09:32.688633 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 609/1235] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jun 21 18:09:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:09:32.691401 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ebe890-788f-43c5-84dd-63c590868549 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:32.738148 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:32.792752 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ebe890-788f-43c5-84dd-63c590868549 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:32.826371 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078bfbe1-4d9c-4a5a-a2c8-71a67b86dc8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:32.827046 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078bfbe1-4d9c-4a5a-a2c8-71a67b86dc8a None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:32.827329 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078bfbe1-4d9c-4a5a-a2c8-71a67b86dc8a None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:32.827609 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078bfbe1-4d9c-4a5a-a2c8-71a67b86dc8a None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:32.828358 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078bfbe1-4d9c-4a5a-a2c8-71a67b86dc8a None admin] RBAC: Adding query filter params (name=service) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:09:32.828698 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078bfbe1-4d9c-4a5a-a2c8-71a67b86dc8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:32.830071 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078bfbe1-4d9c-4a5a-a2c8-71a67b86dc8a None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:32.838063 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 627/1236] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jun 21 18:09:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:09:32.843779 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81effb71-78c3-4940-abcb-625dc3889a7d None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:32.929155 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:32.944273 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81effb71-78c3-4940-abcb-625dc3889a7d None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:32.978095 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55790be6-9161-4400-a945-baea9468c264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:32.978888 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55790be6-9161-4400-a945-baea9468c264 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:32.979176 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55790be6-9161-4400-a945-baea9468c264 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:32.979433 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55790be6-9161-4400-a945-baea9468c264 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:32.989206 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55790be6-9161-4400-a945-baea9468c264 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:32.990663 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55790be6-9161-4400-a945-baea9468c264 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:33.007575 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-55790be6-9161-4400-a945-baea9468c264 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:09:33.007575 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:09:33.007575 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:09:33.007575 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:09:33.007575 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:09:33.007575 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:09:33.007575 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:09:33.007575 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:09:33.007575 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:09:33.007575 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:09:33.007575 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:09:33.007575 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:09:33.007575 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:09:33.007575 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:09:33.007575 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:09:33.008770 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:09:33.008770 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:09:33.008770 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:09:33.008770 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:09:33.008770 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:09:33.008770 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:09:33.008770 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:09:33.008770 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:09:33.008770 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:09:33.008770 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:09:33.008770 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:09:33.008770 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:09:33.008770 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:09:33.008770 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:09:33.008770 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:09:33.008770 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:09:33.008770 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:09:33.009991 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:09:33.009991 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:09:33.009991 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:09:33.009991 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:09:33.009991 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 610/1237] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:09:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:09:33.014673 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078bfbe1-4d9c-4a5a-a2c8-71a67b86dc8a None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:33.116376 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078bfbe1-4d9c-4a5a-a2c8-71a67b86dc8a None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:33.125199 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:33.153244 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9eaf4c-3df8-427a-8e9a-8ff73c5502a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:33.153938 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9eaf4c-3df8-427a-8e9a-8ff73c5502a4 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:33.154221 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9eaf4c-3df8-427a-8e9a-8ff73c5502a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:33.154498 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9eaf4c-3df8-427a-8e9a-8ff73c5502a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:33.155235 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9eaf4c-3df8-427a-8e9a-8ff73c5502a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:09:33.155546 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9eaf4c-3df8-427a-8e9a-8ff73c5502a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:33.156962 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9eaf4c-3df8-427a-8e9a-8ff73c5502a4 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:33.169595 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 628/1238] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:09:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:09:33.175513 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55790be6-9161-4400-a945-baea9468c264 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:33.276051 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55790be6-9161-4400-a945-baea9468c264 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:33.311707 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c81cc7c-7dbb-4f11-9227-71f23a07ec44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:33.312576 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c81cc7c-7dbb-4f11-9227-71f23a07ec44 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:33.312858 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c81cc7c-7dbb-4f11-9227-71f23a07ec44 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:33.313133 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c81cc7c-7dbb-4f11-9227-71f23a07ec44 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:33.315982 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:33.321388 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c81cc7c-7dbb-4f11-9227-71f23a07ec44 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:33.322895 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c81cc7c-7dbb-4f11-9227-71f23a07ec44 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:33.337862 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-6c81cc7c-7dbb-4f11-9227-71f23a07ec44 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 21 18:09:33.337862 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:09:33.337862 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:09:33.337862 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:09:33.337862 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:09:33.337862 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:09:33.337862 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:09:33.337862 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:09:33.337862 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:09:33.337862 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:09:33.337862 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:09:33.337862 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:09:33.337862 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:09:33.337862 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:09:33.337862 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:09:33.337862 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:09:33.339040 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:09:33.339040 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:09:33.339040 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:09:33.339040 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:09:33.339040 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:09:33.339040 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:09:33.339040 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:09:33.339040 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:09:33.339040 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:09:33.339040 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:09:33.339040 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:09:33.339040 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:09:33.339040 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:09:33.339040 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:09:33.339040 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:09:33.339040 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:09:33.339040 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:09:33.339040 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:09:33.340268 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:09:33.340268 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:09:33.340268 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:09:33.340268 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:09:33.340268 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:09:33.340268 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:09:33.340268 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:09:33.340268 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:09:33.340268 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 21 18:09:33.340268 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:09:33.340268 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 611/1239] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:09:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:09:33.345082 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9eaf4c-3df8-427a-8e9a-8ff73c5502a4 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:33.448890 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9eaf4c-3df8-427a-8e9a-8ff73c5502a4 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:33.485508 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d256a99-d18f-470c-a358-88b2db5338c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:33.486228 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d256a99-d18f-470c-a358-88b2db5338c2 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:33.486511 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d256a99-d18f-470c-a358-88b2db5338c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:33.486797 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d256a99-d18f-470c-a358-88b2db5338c2 None admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:33.487632 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d256a99-d18f-470c-a358-88b2db5338c2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:09:33.487942 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d256a99-d18f-470c-a358-88b2db5338c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:33.489507 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d256a99-d18f-470c-a358-88b2db5338c2 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:33.507741 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 629/1240] 10.222.0.23 () {56 vars in 1143 bytes} [Tue Jun 21 18:09:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:09:33.513501 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c81cc7c-7dbb-4f11-9227-71f23a07ec44 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:33.615144 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c81cc7c-7dbb-4f11-9227-71f23a07ec44 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:33.649158 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54011951-b0fd-4c1e-afc7-3c5dcd651b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:33.649800 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54011951-b0fd-4c1e-afc7-3c5dcd651b13 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:33.650125 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54011951-b0fd-4c1e-afc7-3c5dcd651b13 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:33.650356 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54011951-b0fd-4c1e-afc7-3c5dcd651b13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:33.715551 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:33.722796 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54011951-b0fd-4c1e-afc7-3c5dcd651b13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:33.724261 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54011951-b0fd-4c1e-afc7-3c5dcd651b13 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:33.830300 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 612/1241] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:09:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 18:09:33.837339 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d256a99-d18f-470c-a358-88b2db5338c2 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:33.867284 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:33.942471 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d256a99-d18f-470c-a358-88b2db5338c2 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:33.978583 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0277c5-dbca-40e3-a946-3ee24256faab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:33.979438 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0277c5-dbca-40e3-a946-3ee24256faab None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:33.979733 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0277c5-dbca-40e3-a946-3ee24256faab None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:33.980008 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0277c5-dbca-40e3-a946-3ee24256faab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:33.990167 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0277c5-dbca-40e3-a946-3ee24256faab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:33.992433 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0277c5-dbca-40e3-a946-3ee24256faab None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:34.010879 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-8a0277c5-dbca-40e3-a946-3ee24256faab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:09:34.010879 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:09:34.010879 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:09:34.010879 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:09:34.010879 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:09:34.010879 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:09:34.010879 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:09:34.010879 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:09:34.010879 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:09:34.010879 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:09:34.010879 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:09:34.010879 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:09:34.010879 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:09:34.010879 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:09:34.010879 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:09:34.012025 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:09:34.012025 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:09:34.012025 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:09:34.012025 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:09:34.012025 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:09:34.012025 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:09:34.012025 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:09:34.012025 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:09:34.012025 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:09:34.012025 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:09:34.012025 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:09:34.012025 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:09:34.012025 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:09:34.012025 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:09:34.012025 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:09:34.012025 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:09:34.012025 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:09:34.013077 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:09:34.013077 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:09:34.013077 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:09:34.013077 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:09:34.013077 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 630/1242] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:09:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:09:34.018082 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54011951-b0fd-4c1e-afc7-3c5dcd651b13 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:34.112935 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:34.118900 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54011951-b0fd-4c1e-afc7-3c5dcd651b13 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:34.153774 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494aecb1-54f2-443c-9d31-42b0ba5c1568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:34.154443 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494aecb1-54f2-443c-9d31-42b0ba5c1568 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:34.154728 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494aecb1-54f2-443c-9d31-42b0ba5c1568 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:34.155001 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494aecb1-54f2-443c-9d31-42b0ba5c1568 None admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:34.155716 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494aecb1-54f2-443c-9d31-42b0ba5c1568 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:09:34.156022 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494aecb1-54f2-443c-9d31-42b0ba5c1568 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:34.157455 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494aecb1-54f2-443c-9d31-42b0ba5c1568 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:34.169670 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 613/1243] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:09:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:09:34.175389 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0277c5-dbca-40e3-a946-3ee24256faab None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:34.253100 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:34.275707 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0277c5-dbca-40e3-a946-3ee24256faab None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:34.309831 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c33933-cb2a-417d-8b1a-cb8bd6e11ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:34.310627 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c33933-cb2a-417d-8b1a-cb8bd6e11ce1 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:34.310913 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c33933-cb2a-417d-8b1a-cb8bd6e11ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:34.311190 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c33933-cb2a-417d-8b1a-cb8bd6e11ce1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:34.317768 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c33933-cb2a-417d-8b1a-cb8bd6e11ce1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:34.320010 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c33933-cb2a-417d-8b1a-cb8bd6e11ce1 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:34.333676 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-16c33933-cb2a-417d-8b1a-cb8bd6e11ce1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:09:34.333676 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:09:34.333676 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:09:34.333676 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:09:34.333676 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:09:34.333676 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:09:34.333676 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:09:34.333676 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:09:34.333676 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:09:34.333676 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:09:34.333676 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:09:34.333676 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:09:34.333676 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:09:34.333676 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:09:34.333676 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:09:34.335448 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:09:34.335448 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:09:34.335448 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:09:34.335448 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:09:34.335448 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:09:34.335448 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:09:34.335448 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:09:34.335448 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:09:34.335448 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:09:34.335448 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:09:34.335448 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:09:34.335448 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:09:34.335448 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:09:34.335448 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:09:34.335448 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:09:34.335448 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:09:34.335448 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:09:34.337069 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:09:34.337069 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:09:34.337069 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:09:34.337069 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:09:34.337069 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:09:34.337069 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:09:34.337069 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:09:34.337069 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:09:34.337069 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 631/1244] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jun 21 18:09:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:09:34.341344 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494aecb1-54f2-443c-9d31-42b0ba5c1568 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:34.446589 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494aecb1-54f2-443c-9d31-42b0ba5c1568 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:34.483111 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee1823c-1542-4e5a-a8f3-aa21afcf970e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:34.483811 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee1823c-1542-4e5a-a8f3-aa21afcf970e None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:34.484096 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee1823c-1542-4e5a-a8f3-aa21afcf970e None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:34.484373 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee1823c-1542-4e5a-a8f3-aa21afcf970e None admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:34.485184 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee1823c-1542-4e5a-a8f3-aa21afcf970e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:09:34.485492 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee1823c-1542-4e5a-a8f3-aa21afcf970e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:34.486909 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee1823c-1542-4e5a-a8f3-aa21afcf970e None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:34.499689 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 614/1245] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jun 21 18:09:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:09:34.505855 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c33933-cb2a-417d-8b1a-cb8bd6e11ce1 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:34.606711 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c33933-cb2a-417d-8b1a-cb8bd6e11ce1 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:34.641305 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d8f05a-0f8c-4bcc-bb6b-3913963b2f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:34.641954 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d8f05a-0f8c-4bcc-bb6b-3913963b2f8f None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:34.642232 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d8f05a-0f8c-4bcc-bb6b-3913963b2f8f None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:34.642505 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d8f05a-0f8c-4bcc-bb6b-3913963b2f8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:34.643323 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d8f05a-0f8c-4bcc-bb6b-3913963b2f8f None admin] RBAC: Adding query filter params (role.id=acf4e26e0ceb41ddad002dd4edf6373c, scope.project.id=2a4699d1ec70456da5f8e0aa4b1439b7, user.id=938187967efe49be9c5389d1feef81ed) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:09:34.643633 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d8f05a-0f8c-4bcc-bb6b-3913963b2f8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:34.645076 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d8f05a-0f8c-4bcc-bb6b-3913963b2f8f None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:34.651456 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 632/1246] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jun 21 18:09:34 2022] GET /identity/v3/role_assignments?user.id=938187967efe49be9c5389d1feef81ed&scope.project.id=2a4699d1ec70456da5f8e0aa4b1439b7&role.id=acf4e26e0ceb41ddad002dd4edf6373c => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:09:36.775545 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d8db2f-e401-42d0-97cd-1252e128c267 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:36.776130 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d8db2f-e401-42d0-97cd-1252e128c267 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:36.776130 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d8db2f-e401-42d0-97cd-1252e128c267 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:36.777520 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 615/1247] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:09:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:09:36.783115 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3db90f-fb58-491a-a13a-e332d6d78fc5 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:36.783554 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3db90f-fb58-491a-a13a-e332d6d78fc5 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:36.785222 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3db90f-fb58-491a-a13a-e332d6d78fc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:36.789636 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:36.801609 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-bb3db90f-fb58-491a-a13a-e332d6d78fc5 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:09:36.802789 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-bb3db90f-fb58-491a-a13a-e332d6d78fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:09:36.870555 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 633/1248] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:09:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:09:36.894848 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d8db2f-e401-42d0-97cd-1252e128c267 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:36.914468 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:36.998480 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d8db2f-e401-42d0-97cd-1252e128c267 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:37.035037 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8081cf4-4844-407b-8dcf-ab9e314c8c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:37.035677 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8081cf4-4844-407b-8dcf-ab9e314c8c48 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:37.035953 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8081cf4-4844-407b-8dcf-ab9e314c8c48 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:37.036222 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8081cf4-4844-407b-8dcf-ab9e314c8c48 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:37.046955 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8081cf4-4844-407b-8dcf-ab9e314c8c48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:09:37.054523 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-d8081cf4-4844-407b-8dcf-ab9e314c8c48 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:09:37.054523 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:09:37.054523 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:09:37.054523 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:09:37.054523 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:09:37.054523 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:09:37.054523 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:09:37.054523 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:09:37.054523 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:09:37.054523 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:09:37.054523 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:09:37.054523 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:09:37.054523 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:09:37.054523 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:09:37.054523 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:09:37.054523 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:09:37.055705 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:09:37.055705 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:09:37.055705 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:09:37.055705 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:09:37.055705 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:09:37.055705 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:09:37.055705 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:09:37.055705 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:09:37.055705 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:09:37.055705 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:09:37.055705 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:09:37.055705 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:09:37.055705 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:09:37.055705 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:09:37.055705 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:09:37.055705 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:09:37.056601 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:09:37.056601 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:09:37.056601 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:09:37.056601 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:09:37.056601 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:09:37.056601 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:09:37.056601 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:09:37.056601 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:09:37.056601 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:09:37.056601 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:09:37.056601 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:09:37.056601 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:09:37.056601 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 616/1249] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jun 21 18:09:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:09:37.062960 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3db90f-fb58-491a-a13a-e332d6d78fc5 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:37.162810 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3db90f-fb58-491a-a13a-e332d6d78fc5 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:37.191889 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:37.197360 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4467cae-308b-498c-bdc2-0c523784fce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:37.198051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4467cae-308b-498c-bdc2-0c523784fce1 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:37.198365 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4467cae-308b-498c-bdc2-0c523784fce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:37.198644 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4467cae-308b-498c-bdc2-0c523784fce1 None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:37.199381 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4467cae-308b-498c-bdc2-0c523784fce1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:09:37.199684 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4467cae-308b-498c-bdc2-0c523784fce1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:37.201068 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4467cae-308b-498c-bdc2-0c523784fce1 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:37.208912 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 634/1250] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:09:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:09:37.215478 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8081cf4-4844-407b-8dcf-ab9e314c8c48 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:37.293897 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:37.315674 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8081cf4-4844-407b-8dcf-ab9e314c8c48 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:37.349113 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34c1b8f-0264-4103-ac0a-f10ea7ede9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:37.349919 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c1b8f-0264-4103-ac0a-f10ea7ede9c6 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:37.350713 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c1b8f-0264-4103-ac0a-f10ea7ede9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:37.350992 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c1b8f-0264-4103-ac0a-f10ea7ede9c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:37.361321 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34c1b8f-0264-4103-ac0a-f10ea7ede9c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:37.362816 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34c1b8f-0264-4103-ac0a-f10ea7ede9c6 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:37.379842 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-b34c1b8f-0264-4103-ac0a-f10ea7ede9c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:09:37.379842 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:09:37.379842 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:09:37.379842 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:09:37.379842 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:09:37.379842 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:09:37.379842 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:09:37.379842 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:09:37.379842 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:09:37.379842 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:09:37.379842 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:09:37.379842 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:09:37.379842 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:09:37.379842 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:09:37.379842 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:09:37.381807 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:09:37.381807 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:09:37.381807 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:09:37.381807 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:09:37.381807 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:09:37.381807 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:09:37.381807 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:09:37.381807 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:09:37.381807 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:09:37.381807 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:09:37.381807 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:09:37.381807 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:09:37.381807 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:09:37.381807 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:09:37.381807 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:09:37.381807 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:09:37.381807 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:09:37.383069 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:09:37.383069 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:09:37.383069 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:09:37.383069 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:09:37.383069 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 617/1251] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:09:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:09:37.387834 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4467cae-308b-498c-bdc2-0c523784fce1 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:37.491421 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4467cae-308b-498c-bdc2-0c523784fce1 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:37.527675 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57de929f-0b7d-4f6b-8a80-4e5572c885a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:37.528343 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57de929f-0b7d-4f6b-8a80-4e5572c885a8 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:37.528658 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57de929f-0b7d-4f6b-8a80-4e5572c885a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:37.528938 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57de929f-0b7d-4f6b-8a80-4e5572c885a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:37.529659 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57de929f-0b7d-4f6b-8a80-4e5572c885a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:09:37.529968 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57de929f-0b7d-4f6b-8a80-4e5572c885a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:37.531335 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57de929f-0b7d-4f6b-8a80-4e5572c885a8 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:37.544510 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 635/1252] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:09:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:09:37.550224 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34c1b8f-0264-4103-ac0a-f10ea7ede9c6 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:37.651212 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34c1b8f-0264-4103-ac0a-f10ea7ede9c6 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:37.676722 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:37.685100 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1617e2e1-1210-4cc3-9484-3ae0353c7fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:37.685911 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1617e2e1-1210-4cc3-9484-3ae0353c7fb1 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:37.686198 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1617e2e1-1210-4cc3-9484-3ae0353c7fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:37.686477 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1617e2e1-1210-4cc3-9484-3ae0353c7fb1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:37.695008 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1617e2e1-1210-4cc3-9484-3ae0353c7fb1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:37.696463 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1617e2e1-1210-4cc3-9484-3ae0353c7fb1 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:37.710719 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-1617e2e1-1210-4cc3-9484-3ae0353c7fb1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 21 18:09:37.710719 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:09:37.710719 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:09:37.710719 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:09:37.710719 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:09:37.710719 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:09:37.710719 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:09:37.710719 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:09:37.710719 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:09:37.710719 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:09:37.710719 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:09:37.710719 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:09:37.710719 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:09:37.710719 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:09:37.710719 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:09:37.710719 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:09:37.711821 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:09:37.711821 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:09:37.711821 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:09:37.711821 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:09:37.711821 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:09:37.711821 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:09:37.711821 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:09:37.711821 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:09:37.711821 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:09:37.711821 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:09:37.711821 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:09:37.711821 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:09:37.711821 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:09:37.711821 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:09:37.711821 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:09:37.711821 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:09:37.711821 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:09:37.711821 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:09:37.713068 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:09:37.713068 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:09:37.713068 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:09:37.713068 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:09:37.713068 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:09:37.713068 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:09:37.713068 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:09:37.713068 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:09:37.713068 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 21 18:09:37.713068 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:09:37.713068 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 618/1253] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:09:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:09:37.717763 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57de929f-0b7d-4f6b-8a80-4e5572c885a8 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:37.771473 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:37.815157 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57de929f-0b7d-4f6b-8a80-4e5572c885a8 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:37.848494 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ba3807-72f4-4115-a8a8-b4418dda3bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:37.849196 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba3807-72f4-4115-a8a8-b4418dda3bdd None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:37.849483 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba3807-72f4-4115-a8a8-b4418dda3bdd None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:37.849763 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba3807-72f4-4115-a8a8-b4418dda3bdd None admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:37.851237 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ba3807-72f4-4115-a8a8-b4418dda3bdd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:09:37.851551 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ba3807-72f4-4115-a8a8-b4418dda3bdd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:37.853293 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ba3807-72f4-4115-a8a8-b4418dda3bdd None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:37.872330 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 636/1254] 10.222.0.23 () {56 vars in 1143 bytes} [Tue Jun 21 18:09:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:09:37.878047 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1617e2e1-1210-4cc3-9484-3ae0353c7fb1 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:37.980000 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1617e2e1-1210-4cc3-9484-3ae0353c7fb1 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:38.013777 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98b7ea1-2baf-4e3b-bddf-2021b6250265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:38.014431 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b7ea1-2baf-4e3b-bddf-2021b6250265 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:38.014712 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b7ea1-2baf-4e3b-bddf-2021b6250265 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:38.014983 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b7ea1-2baf-4e3b-bddf-2021b6250265 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:38.072159 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:38.083889 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98b7ea1-2baf-4e3b-bddf-2021b6250265 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:38.085905 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98b7ea1-2baf-4e3b-bddf-2021b6250265 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:38.189120 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 619/1255] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:09:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 18:09:38.195744 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ba3807-72f4-4115-a8a8-b4418dda3bdd None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:38.294636 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ba3807-72f4-4115-a8a8-b4418dda3bdd None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:38.309181 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:38.330385 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a130eaa3-f781-4c0f-ba9c-579e0edd5bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:38.331139 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a130eaa3-f781-4c0f-ba9c-579e0edd5bb9 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:38.331427 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a130eaa3-f781-4c0f-ba9c-579e0edd5bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:38.331706 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a130eaa3-f781-4c0f-ba9c-579e0edd5bb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:38.342415 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a130eaa3-f781-4c0f-ba9c-579e0edd5bb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:38.343862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a130eaa3-f781-4c0f-ba9c-579e0edd5bb9 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:38.362226 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-a130eaa3-f781-4c0f-ba9c-579e0edd5bb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:09:38.362226 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:09:38.362226 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:09:38.362226 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:09:38.362226 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:09:38.362226 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:09:38.362226 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:09:38.362226 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:09:38.362226 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:09:38.362226 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:09:38.362226 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:09:38.362226 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:09:38.362226 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:09:38.362226 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:09:38.362226 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:09:38.363320 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:09:38.363320 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:09:38.363320 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:09:38.363320 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:09:38.363320 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:09:38.363320 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:09:38.363320 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:09:38.363320 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:09:38.363320 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:09:38.363320 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:09:38.363320 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:09:38.363320 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:09:38.363320 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:09:38.363320 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:09:38.363320 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:09:38.363320 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:09:38.363320 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:09:38.364270 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:09:38.364270 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:09:38.364270 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:09:38.364270 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:09:38.364270 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 637/1256] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:09:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:09:38.370228 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98b7ea1-2baf-4e3b-bddf-2021b6250265 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:38.456827 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:38.468231 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98b7ea1-2baf-4e3b-bddf-2021b6250265 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:38.501944 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895c4c50-ac2c-46bd-bfa0-a03852fbd18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:38.502611 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c4c50-ac2c-46bd-bfa0-a03852fbd18e None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:38.502900 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c4c50-ac2c-46bd-bfa0-a03852fbd18e None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:38.503179 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c4c50-ac2c-46bd-bfa0-a03852fbd18e None admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:38.503901 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895c4c50-ac2c-46bd-bfa0-a03852fbd18e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:09:38.504210 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895c4c50-ac2c-46bd-bfa0-a03852fbd18e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:38.505658 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895c4c50-ac2c-46bd-bfa0-a03852fbd18e None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:38.517541 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 620/1257] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:09:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:09:38.523788 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a130eaa3-f781-4c0f-ba9c-579e0edd5bb9 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:38.627597 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a130eaa3-f781-4c0f-ba9c-579e0edd5bb9 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:38.661562 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a177d8e7-e641-4278-8822-20631f8ab5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:38.662819 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a177d8e7-e641-4278-8822-20631f8ab5f2 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:38.663116 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a177d8e7-e641-4278-8822-20631f8ab5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:38.663388 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a177d8e7-e641-4278-8822-20631f8ab5f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:38.670226 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a177d8e7-e641-4278-8822-20631f8ab5f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:38.671673 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a177d8e7-e641-4278-8822-20631f8ab5f2 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:38.686495 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-a177d8e7-e641-4278-8822-20631f8ab5f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:09:38.686495 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:09:38.686495 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:09:38.686495 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:09:38.686495 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:09:38.686495 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:09:38.686495 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:09:38.686495 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:09:38.686495 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:09:38.686495 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:09:38.686495 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:09:38.686495 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:09:38.686495 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:09:38.686495 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:09:38.686495 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:09:38.687776 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:09:38.687776 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:09:38.687776 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:09:38.687776 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:09:38.687776 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:09:38.687776 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:09:38.687776 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:09:38.687776 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:09:38.687776 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:09:38.687776 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:09:38.687776 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:09:38.687776 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:09:38.687776 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:09:38.687776 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:09:38.687776 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:09:38.687776 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:09:38.687776 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:09:38.688915 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:09:38.688915 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:09:38.688915 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:09:38.688915 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:09:38.688915 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:09:38.688915 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:09:38.688915 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:09:38.688915 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:09:38.688915 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 638/1258] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jun 21 18:09:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:09:38.693604 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895c4c50-ac2c-46bd-bfa0-a03852fbd18e None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:38.799942 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895c4c50-ac2c-46bd-bfa0-a03852fbd18e None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:38.834029 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb2826d-cb23-48f6-afef-8c5b6e79e683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:38.835812 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb2826d-cb23-48f6-afef-8c5b6e79e683 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:38.836097 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb2826d-cb23-48f6-afef-8c5b6e79e683 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:38.836372 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb2826d-cb23-48f6-afef-8c5b6e79e683 None admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:38.837186 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb2826d-cb23-48f6-afef-8c5b6e79e683 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:09:38.837493 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb2826d-cb23-48f6-afef-8c5b6e79e683 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:38.838932 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb2826d-cb23-48f6-afef-8c5b6e79e683 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:38.851532 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 621/1259] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jun 21 18:09:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:09:38.858140 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a177d8e7-e641-4278-8822-20631f8ab5f2 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:38.885289 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:38.965122 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a177d8e7-e641-4278-8822-20631f8ab5f2 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:39.001871 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59267539-e800-4ce1-8b01-173262c1b09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:39.002540 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59267539-e800-4ce1-8b01-173262c1b09d None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:39.002819 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59267539-e800-4ce1-8b01-173262c1b09d None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:39.003094 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59267539-e800-4ce1-8b01-173262c1b09d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:39.003918 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59267539-e800-4ce1-8b01-173262c1b09d None admin] RBAC: Adding query filter params (role.id=745ed9065a8143e384ebbd694b29635c, scope.project.id=2a4699d1ec70456da5f8e0aa4b1439b7, user.id=938187967efe49be9c5389d1feef81ed) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:09:39.004226 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59267539-e800-4ce1-8b01-173262c1b09d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:39.005675 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59267539-e800-4ce1-8b01-173262c1b09d None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:39.012790 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 639/1260] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jun 21 18:09:38 2022] GET /identity/v3/role_assignments?user.id=938187967efe49be9c5389d1feef81ed&scope.project.id=2a4699d1ec70456da5f8e0aa4b1439b7&role.id=745ed9065a8143e384ebbd694b29635c => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:09:41.093379 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6070904-4c09-44c9-91d0-48c485b2067b None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:41.093961 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6070904-4c09-44c9-91d0-48c485b2067b None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:41.093961 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6070904-4c09-44c9-91d0-48c485b2067b None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:41.095355 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 622/1261] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:09:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:09:41.101327 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c602b9-1f30-4dd3-9d5c-1510a5bc762e None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:41.101786 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c602b9-1f30-4dd3-9d5c-1510a5bc762e None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:41.101879 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c602b9-1f30-4dd3-9d5c-1510a5bc762e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:41.119654 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-77c602b9-1f30-4dd3-9d5c-1510a5bc762e None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:09:41.120894 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-77c602b9-1f30-4dd3-9d5c-1510a5bc762e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:09:41.188875 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 640/1262] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:09:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:09:41.212778 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6070904-4c09-44c9-91d0-48c485b2067b None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:41.223538 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:41.314498 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6070904-4c09-44c9-91d0-48c485b2067b None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:41.349140 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65f1315-9fd3-433e-89e1-8202c2066f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:41.351037 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f1315-9fd3-433e-89e1-8202c2066f50 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:41.351330 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f1315-9fd3-433e-89e1-8202c2066f50 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:41.351612 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f1315-9fd3-433e-89e1-8202c2066f50 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:41.361946 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65f1315-9fd3-433e-89e1-8202c2066f50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:09:41.370183 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-e65f1315-9fd3-433e-89e1-8202c2066f50 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:09:41.370183 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:09:41.370183 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:09:41.370183 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:09:41.370183 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:09:41.370183 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:09:41.370183 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:09:41.370183 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:09:41.370183 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:09:41.370183 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:09:41.370183 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:09:41.370183 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:09:41.370183 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:09:41.370183 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:09:41.370183 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:09:41.370183 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:09:41.371213 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:09:41.371213 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:09:41.371213 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:09:41.371213 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:09:41.371213 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:09:41.371213 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:09:41.371213 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:09:41.371213 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:09:41.371213 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:09:41.371213 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:09:41.371213 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:09:41.371213 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:09:41.371213 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:09:41.371213 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:09:41.371213 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:09:41.371213 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:09:41.372265 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:09:41.372265 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:09:41.372265 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:09:41.372265 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:09:41.372265 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:09:41.372265 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:09:41.372265 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:09:41.372265 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:09:41.372265 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:09:41.372265 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:09:41.372265 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:09:41.372265 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:09:41.372265 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 623/1263] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jun 21 18:09:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:09:41.377601 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c602b9-1f30-4dd3-9d5c-1510a5bc762e None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:41.396060 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:41.479949 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c602b9-1f30-4dd3-9d5c-1510a5bc762e None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:41.515403 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25e07e0-7587-469c-a0c8-b689d69fb998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:41.516080 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e07e0-7587-469c-a0c8-b689d69fb998 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:41.516363 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e07e0-7587-469c-a0c8-b689d69fb998 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:41.516666 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e07e0-7587-469c-a0c8-b689d69fb998 None admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:41.517413 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25e07e0-7587-469c-a0c8-b689d69fb998 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:09:41.517720 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25e07e0-7587-469c-a0c8-b689d69fb998 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:41.519090 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25e07e0-7587-469c-a0c8-b689d69fb998 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:41.532927 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 641/1264] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:09:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:09:41.534159 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65f1315-9fd3-433e-89e1-8202c2066f50 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:41.601218 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:41.634515 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65f1315-9fd3-433e-89e1-8202c2066f50 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:41.668379 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d149cc44-8635-40db-a669-78aa75562013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:41.669249 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d149cc44-8635-40db-a669-78aa75562013 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:41.669534 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d149cc44-8635-40db-a669-78aa75562013 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:41.669811 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d149cc44-8635-40db-a669-78aa75562013 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:41.680514 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d149cc44-8635-40db-a669-78aa75562013 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:41.682019 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d149cc44-8635-40db-a669-78aa75562013 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:41.700229 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-d149cc44-8635-40db-a669-78aa75562013 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:09:41.700229 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:09:41.700229 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:09:41.700229 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:09:41.700229 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:09:41.700229 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:09:41.700229 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:09:41.700229 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:09:41.700229 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:09:41.700229 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:09:41.700229 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:09:41.700229 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:09:41.700229 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:09:41.700229 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:09:41.700229 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:09:41.701399 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:09:41.701399 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:09:41.701399 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:09:41.701399 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:09:41.701399 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:09:41.701399 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:09:41.701399 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:09:41.701399 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:09:41.701399 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:09:41.701399 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:09:41.701399 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:09:41.701399 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:09:41.701399 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:09:41.701399 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:09:41.701399 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:09:41.701399 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:09:41.701399 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:09:41.702539 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:09:41.702539 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:09:41.702539 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:09:41.702539 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:09:41.702539 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 624/1265] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:09:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:09:41.707854 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25e07e0-7587-469c-a0c8-b689d69fb998 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:41.790885 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:41.810058 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25e07e0-7587-469c-a0c8-b689d69fb998 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:41.846973 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616b961b-cdd9-4de2-b6db-d8e09769b1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:41.847913 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b961b-cdd9-4de2-b6db-d8e09769b1d3 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:41.848197 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b961b-cdd9-4de2-b6db-d8e09769b1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:41.848507 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b961b-cdd9-4de2-b6db-d8e09769b1d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:41.849247 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616b961b-cdd9-4de2-b6db-d8e09769b1d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:09:41.849696 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616b961b-cdd9-4de2-b6db-d8e09769b1d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:41.851100 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616b961b-cdd9-4de2-b6db-d8e09769b1d3 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:41.864737 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 642/1266] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:09:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:09:41.871621 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d149cc44-8635-40db-a669-78aa75562013 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:41.975059 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d149cc44-8635-40db-a669-78aa75562013 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:41.997655 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:42.012951 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b3bfd3-555f-4931-8d6e-dba5fd233129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:42.013771 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3bfd3-555f-4931-8d6e-dba5fd233129 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:42.014057 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3bfd3-555f-4931-8d6e-dba5fd233129 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:42.014331 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3bfd3-555f-4931-8d6e-dba5fd233129 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:42.023805 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b3bfd3-555f-4931-8d6e-dba5fd233129 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:42.025310 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b3bfd3-555f-4931-8d6e-dba5fd233129 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:42.040100 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-70b3bfd3-555f-4931-8d6e-dba5fd233129 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 21 18:09:42.040100 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:09:42.040100 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:09:42.040100 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:09:42.040100 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:09:42.040100 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:09:42.040100 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:09:42.040100 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:09:42.040100 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:09:42.040100 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:09:42.040100 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:09:42.040100 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:09:42.040100 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:09:42.040100 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:09:42.040100 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:09:42.040100 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:09:42.043735 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:09:42.043735 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:09:42.043735 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:09:42.043735 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:09:42.043735 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:09:42.043735 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:09:42.043735 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:09:42.043735 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:09:42.043735 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:09:42.043735 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:09:42.043735 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:09:42.043735 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:09:42.043735 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:09:42.043735 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:09:42.043735 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:09:42.043735 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:09:42.043735 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:09:42.043735 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:09:42.044959 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:09:42.044959 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:09:42.044959 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:09:42.044959 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:09:42.044959 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:09:42.044959 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:09:42.044959 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:09:42.044959 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:09:42.044959 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 21 18:09:42.044959 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:09:42.044959 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 625/1267] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:09:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:09:42.047443 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616b961b-cdd9-4de2-b6db-d8e09769b1d3 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:42.155217 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616b961b-cdd9-4de2-b6db-d8e09769b1d3 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:42.190889 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee6937-dd67-4d40-9843-6533073ee08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:42.191565 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee6937-dd67-4d40-9843-6533073ee08b None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:42.191852 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee6937-dd67-4d40-9843-6533073ee08b None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:42.192131 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee6937-dd67-4d40-9843-6533073ee08b None admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:42.193056 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ee6937-dd67-4d40-9843-6533073ee08b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:09:42.199077 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ee6937-dd67-4d40-9843-6533073ee08b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:42.199077 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ee6937-dd67-4d40-9843-6533073ee08b None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:42.203728 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:42.213086 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 643/1268] 10.222.0.23 () {56 vars in 1143 bytes} [Tue Jun 21 18:09:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:09:42.219054 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b3bfd3-555f-4931-8d6e-dba5fd233129 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:42.324126 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b3bfd3-555f-4931-8d6e-dba5fd233129 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:42.363383 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a43616-0917-41ec-9bbc-44480bcb9af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:42.364785 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a43616-0917-41ec-9bbc-44480bcb9af6 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:42.365265 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a43616-0917-41ec-9bbc-44480bcb9af6 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:42.365799 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a43616-0917-41ec-9bbc-44480bcb9af6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:42.377308 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a43616-0917-41ec-9bbc-44480bcb9af6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:42.378770 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a43616-0917-41ec-9bbc-44480bcb9af6 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:42.396290 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-38a43616-0917-41ec-9bbc-44480bcb9af6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:09:42.396290 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:09:42.396290 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:09:42.396290 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:09:42.396290 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:09:42.396290 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:09:42.396290 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:09:42.396290 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:09:42.396290 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:09:42.396290 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:09:42.396290 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:09:42.396290 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:09:42.396290 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:09:42.396290 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:09:42.396290 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:09:42.397278 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:09:42.397278 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:09:42.397278 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:09:42.397278 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:09:42.397278 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:09:42.397278 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:09:42.397278 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:09:42.397278 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:09:42.397278 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:09:42.397278 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:09:42.397278 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:09:42.397278 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:09:42.397278 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:09:42.397278 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:09:42.397278 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:09:42.397278 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:09:42.397278 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:09:42.398270 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:09:42.398270 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:09:42.398270 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:09:42.398270 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:09:42.398270 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 626/1269] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:09:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:09:42.404128 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee6937-dd67-4d40-9843-6533073ee08b None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:42.514057 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee6937-dd67-4d40-9843-6533073ee08b None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:42.548367 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eb478e-006c-4f51-b095-9163ba6cb8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:42.549056 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eb478e-006c-4f51-b095-9163ba6cb8b4 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:42.549346 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eb478e-006c-4f51-b095-9163ba6cb8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:42.549626 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eb478e-006c-4f51-b095-9163ba6cb8b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:42.550355 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eb478e-006c-4f51-b095-9163ba6cb8b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:09:42.550662 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eb478e-006c-4f51-b095-9163ba6cb8b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:42.552028 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eb478e-006c-4f51-b095-9163ba6cb8b4 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:42.564159 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 644/1270] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:09:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:09:42.570564 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a43616-0917-41ec-9bbc-44480bcb9af6 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:42.604012 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:42.672223 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a43616-0917-41ec-9bbc-44480bcb9af6 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:42.712791 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611df80a-d616-4658-b884-a98a2623f46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:42.713613 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611df80a-d616-4658-b884-a98a2623f46e None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:42.713897 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611df80a-d616-4658-b884-a98a2623f46e None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:42.714172 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611df80a-d616-4658-b884-a98a2623f46e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:42.721926 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611df80a-d616-4658-b884-a98a2623f46e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:42.723401 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611df80a-d616-4658-b884-a98a2623f46e None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:42.739164 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-611df80a-d616-4658-b884-a98a2623f46e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:09:42.739164 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:09:42.739164 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:09:42.739164 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:09:42.739164 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:09:42.739164 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:09:42.739164 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:09:42.739164 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:09:42.739164 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:09:42.739164 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:09:42.739164 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:09:42.739164 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:09:42.739164 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:09:42.739164 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:09:42.739164 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:09:42.740382 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:09:42.740382 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:09:42.740382 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:09:42.740382 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:09:42.740382 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:09:42.740382 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:09:42.740382 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:09:42.740382 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:09:42.740382 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:09:42.740382 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:09:42.740382 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:09:42.740382 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:09:42.740382 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:09:42.740382 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:09:42.740382 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:09:42.740382 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:09:42.740382 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:09:42.742102 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:09:42.742102 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:09:42.742102 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:09:42.742102 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:09:42.742102 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:09:42.742102 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:09:42.742102 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:09:42.742102 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:09:42.742102 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 627/1271] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jun 21 18:09:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:09:42.747347 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eb478e-006c-4f51-b095-9163ba6cb8b4 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:42.816141 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:42.862739 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eb478e-006c-4f51-b095-9163ba6cb8b4 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:42.896009 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8ddeb8-3678-4862-b4f7-50d3ad5bbb7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:42.908337 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ddeb8-3678-4862-b4f7-50d3ad5bbb7b None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:42.908337 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ddeb8-3678-4862-b4f7-50d3ad5bbb7b None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:42.908337 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ddeb8-3678-4862-b4f7-50d3ad5bbb7b None admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:42.908337 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8ddeb8-3678-4862-b4f7-50d3ad5bbb7b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:09:42.908337 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8ddeb8-3678-4862-b4f7-50d3ad5bbb7b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:42.908337 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8ddeb8-3678-4862-b4f7-50d3ad5bbb7b None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:42.911794 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 645/1272] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jun 21 18:09:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:09:42.917954 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611df80a-d616-4658-b884-a98a2623f46e None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:43.017975 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:43.027606 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611df80a-d616-4658-b884-a98a2623f46e None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:43.062096 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee881c5-e8f9-436e-a192-3f3ec4659e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:43.063011 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee881c5-e8f9-436e-a192-3f3ec4659e53 None admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:43.063292 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee881c5-e8f9-436e-a192-3f3ec4659e53 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:43.063568 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee881c5-e8f9-436e-a192-3f3ec4659e53 None admin] PATH_INFO: `/v3/projects/2a4699d1ec70456da5f8e0aa4b1439b7/users/938187967efe49be9c5389d1feef81ed/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:43.065541 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee881c5-e8f9-436e-a192-3f3ec4659e53 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a4699d1ec70456da5f8e0aa4b1439b7, user_id=938187967efe49be9c5389d1feef81ed, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:43.067010 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee881c5-e8f9-436e-a192-3f3ec4659e53 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:43.139520 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 628/1273] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jun 21 18:09:42 2022] PUT /identity/v3/projects/2a4699d1ec70456da5f8e0aa4b1439b7/users/938187967efe49be9c5389d1feef81ed/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:09:45.160589 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8a747-4924-4aa0-8100-c138b0c836cf None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:45.161171 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8a747-4924-4aa0-8100-c138b0c836cf None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:45.161171 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8a747-4924-4aa0-8100-c138b0c836cf None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:45.162606 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 646/1274] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:09:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:09:45.168365 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bcc181-c4c1-4e3e-b05b-0484247272fa None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:45.174911 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bcc181-c4c1-4e3e-b05b-0484247272fa None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:45.174911 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bcc181-c4c1-4e3e-b05b-0484247272fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:45.186446 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-93bcc181-c4c1-4e3e-b05b-0484247272fa None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:09:45.187674 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-93bcc181-c4c1-4e3e-b05b-0484247272fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:09:45.256076 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 629/1275] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:09:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:09:45.279562 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe8a747-4924-4aa0-8100-c138b0c836cf None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:45.383198 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe8a747-4924-4aa0-8100-c138b0c836cf None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:45.401049 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:45.418681 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518ff55-bbf3-434c-b139-f27f99d10161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:45.419404 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518ff55-bbf3-434c-b139-f27f99d10161 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:45.419688 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518ff55-bbf3-434c-b139-f27f99d10161 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:45.419964 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518ff55-bbf3-434c-b139-f27f99d10161 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:45.430526 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2518ff55-bbf3-434c-b139-f27f99d10161 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:09:45.438273 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-2518ff55-bbf3-434c-b139-f27f99d10161 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:09:45.438273 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:09:45.438273 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:09:45.438273 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:09:45.438273 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:09:45.438273 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:09:45.438273 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:09:45.438273 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:09:45.438273 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:09:45.438273 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:09:45.438273 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:09:45.438273 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:09:45.438273 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:09:45.438273 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:09:45.438273 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:09:45.438273 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:09:45.439590 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:09:45.439590 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:09:45.439590 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:09:45.439590 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:09:45.439590 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:09:45.439590 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:09:45.439590 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:09:45.439590 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:09:45.439590 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:09:45.439590 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:09:45.439590 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:09:45.439590 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:09:45.439590 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:09:45.439590 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:09:45.439590 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:09:45.439590 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:09:45.440745 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:09:45.440745 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:09:45.440745 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:09:45.440745 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:09:45.440745 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:09:45.440745 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:09:45.440745 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:09:45.440745 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:09:45.440745 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:09:45.440745 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:09:45.440745 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:09:45.440745 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:09:45.440745 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 647/1276] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jun 21 18:09:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:09:45.446047 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bcc181-c4c1-4e3e-b05b-0484247272fa None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:45.530285 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:45.548245 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bcc181-c4c1-4e3e-b05b-0484247272fa None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:45.585848 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9800a8a-1df5-4504-8b9e-c5d8e0498287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:45.586523 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9800a8a-1df5-4504-8b9e-c5d8e0498287 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:45.586901 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9800a8a-1df5-4504-8b9e-c5d8e0498287 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:45.587159 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9800a8a-1df5-4504-8b9e-c5d8e0498287 None admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:45.588137 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9800a8a-1df5-4504-8b9e-c5d8e0498287 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:09:45.588538 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9800a8a-1df5-4504-8b9e-c5d8e0498287 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:45.594127 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9800a8a-1df5-4504-8b9e-c5d8e0498287 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:45.599002 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 630/1277] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jun 21 18:09:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:09:45.605517 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518ff55-bbf3-434c-b139-f27f99d10161 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:45.713888 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518ff55-bbf3-434c-b139-f27f99d10161 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:45.749136 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53867108-5106-42cc-8f38-f3446001fdd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:45.749921 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53867108-5106-42cc-8f38-f3446001fdd1 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:45.750201 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53867108-5106-42cc-8f38-f3446001fdd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:45.750472 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53867108-5106-42cc-8f38-f3446001fdd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:45.761758 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53867108-5106-42cc-8f38-f3446001fdd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:45.763197 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53867108-5106-42cc-8f38-f3446001fdd1 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:45.781456 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-53867108-5106-42cc-8f38-f3446001fdd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:09:45.781456 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:09:45.781456 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:09:45.781456 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:09:45.781456 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:09:45.781456 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:09:45.781456 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:09:45.781456 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:09:45.781456 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:09:45.781456 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:09:45.781456 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:09:45.781456 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:09:45.781456 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:09:45.781456 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:09:45.781456 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:09:45.782602 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:09:45.782602 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:09:45.782602 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:09:45.782602 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:09:45.782602 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:09:45.782602 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:09:45.782602 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:09:45.782602 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:09:45.782602 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:09:45.782602 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:09:45.782602 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:09:45.782602 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:09:45.782602 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:09:45.782602 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:09:45.782602 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:09:45.782602 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:09:45.782602 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:09:45.783705 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:09:45.783705 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:09:45.783705 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:09:45.783705 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:09:45.783705 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 648/1278] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:09:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:09:45.789278 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9800a8a-1df5-4504-8b9e-c5d8e0498287 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:45.895680 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9800a8a-1df5-4504-8b9e-c5d8e0498287 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:45.929493 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cba9a2-8fb8-467f-b061-61c7c7ff9436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:45.930155 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cba9a2-8fb8-467f-b061-61c7c7ff9436 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:45.930438 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cba9a2-8fb8-467f-b061-61c7c7ff9436 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:45.930714 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cba9a2-8fb8-467f-b061-61c7c7ff9436 None admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:45.931435 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cba9a2-8fb8-467f-b061-61c7c7ff9436 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:09:45.931741 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cba9a2-8fb8-467f-b061-61c7c7ff9436 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:45.933126 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cba9a2-8fb8-467f-b061-61c7c7ff9436 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:45.945675 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 631/1279] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:09:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:09:45.951540 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53867108-5106-42cc-8f38-f3446001fdd1 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:45.985283 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:46.061690 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53867108-5106-42cc-8f38-f3446001fdd1 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:46.103015 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12dfd04-3c1c-4834-bc72-334750c20bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:46.103906 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12dfd04-3c1c-4834-bc72-334750c20bda None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:46.104191 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12dfd04-3c1c-4834-bc72-334750c20bda None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:46.104499 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12dfd04-3c1c-4834-bc72-334750c20bda None admin] PATH_INFO: `/v3/users/placement` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:46.112807 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12dfd04-3c1c-4834-bc72-334750c20bda None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:46.114251 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12dfd04-3c1c-4834-bc72-334750c20bda None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:46.131158 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-a12dfd04-3c1c-4834-bc72-334750c20bda None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 21 18:09:46.131158 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:09:46.131158 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:09:46.131158 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:09:46.131158 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:09:46.131158 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:09:46.131158 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:09:46.131158 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:09:46.131158 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:09:46.131158 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:09:46.131158 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:09:46.131158 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:09:46.131158 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:09:46.131158 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:09:46.131158 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:09:46.131158 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:09:46.132976 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:09:46.132976 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:09:46.132976 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:09:46.132976 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:09:46.132976 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:09:46.132976 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:09:46.132976 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:09:46.132976 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:09:46.132976 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:09:46.132976 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:09:46.132976 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:09:46.132976 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:09:46.132976 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:09:46.132976 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:09:46.132976 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:09:46.132976 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:09:46.132976 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:09:46.132976 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:09:46.133960 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:09:46.133960 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:09:46.133960 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:09:46.133960 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:09:46.133960 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:09:46.133960 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:09:46.133960 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:09:46.133960 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:09:46.133960 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 21 18:09:46.133960 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:09:46.133960 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 649/1280] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:09:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:09:46.138571 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cba9a2-8fb8-467f-b061-61c7c7ff9436 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:46.142599 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:46.244095 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cba9a2-8fb8-467f-b061-61c7c7ff9436 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:46.278358 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfde1f5e-ac63-4e4a-ae14-d0b54de17490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:46.279059 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde1f5e-ac63-4e4a-ae14-d0b54de17490 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:46.279375 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde1f5e-ac63-4e4a-ae14-d0b54de17490 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:46.279653 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde1f5e-ac63-4e4a-ae14-d0b54de17490 None admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:46.280572 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfde1f5e-ac63-4e4a-ae14-d0b54de17490 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:09:46.280882 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfde1f5e-ac63-4e4a-ae14-d0b54de17490 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:46.282311 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfde1f5e-ac63-4e4a-ae14-d0b54de17490 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:46.301599 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 632/1281] 10.222.0.23 () {56 vars in 1143 bytes} [Tue Jun 21 18:09:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:09:46.308166 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12dfd04-3c1c-4834-bc72-334750c20bda None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:46.409586 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:46.420232 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12dfd04-3c1c-4834-bc72-334750c20bda None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:46.454834 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc7b3fa-de13-4bd7-91a3-11c5b093ac7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:46.455493 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc7b3fa-de13-4bd7-91a3-11c5b093ac7c None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:46.455777 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc7b3fa-de13-4bd7-91a3-11c5b093ac7c None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:46.456055 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc7b3fa-de13-4bd7-91a3-11c5b093ac7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:46.535553 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc7b3fa-de13-4bd7-91a3-11c5b093ac7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:46.536965 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc7b3fa-de13-4bd7-91a3-11c5b093ac7c None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:46.630445 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:46.641729 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 650/1282] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jun 21 18:09:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 18:09:46.648186 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfde1f5e-ac63-4e4a-ae14-d0b54de17490 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:46.712557 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:46.751831 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfde1f5e-ac63-4e4a-ae14-d0b54de17490 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:46.785314 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd9ee7-5c93-415d-a71f-3b11f6939e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:46.786998 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd9ee7-5c93-415d-a71f-3b11f6939e59 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:46.786998 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd9ee7-5c93-415d-a71f-3b11f6939e59 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:46.786998 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd9ee7-5c93-415d-a71f-3b11f6939e59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:46.798218 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bd9ee7-5c93-415d-a71f-3b11f6939e59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:46.799755 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bd9ee7-5c93-415d-a71f-3b11f6939e59 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:46.818331 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-79bd9ee7-5c93-415d-a71f-3b11f6939e59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:09:46.818331 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:09:46.818331 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:09:46.818331 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:09:46.818331 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:09:46.818331 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:09:46.818331 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:09:46.818331 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:09:46.818331 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:09:46.818331 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:09:46.818331 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:09:46.818331 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:09:46.818331 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:09:46.818331 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:09:46.818331 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:09:46.822956 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:09:46.822956 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:09:46.822956 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:09:46.822956 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:09:46.822956 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:09:46.822956 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:09:46.822956 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:09:46.822956 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:09:46.822956 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:09:46.822956 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:09:46.822956 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:09:46.822956 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:09:46.822956 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:09:46.822956 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:09:46.822956 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:09:46.822956 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:09:46.822956 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:09:46.824112 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:09:46.824112 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:09:46.824112 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:09:46.824112 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:09:46.824112 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 633/1283] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:09:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:09:46.825913 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc7b3fa-de13-4bd7-91a3-11c5b093ac7c None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:46.930896 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc7b3fa-de13-4bd7-91a3-11c5b093ac7c None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:46.966072 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a40495-9eee-437d-bc4e-fe976c5c5d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:46.966748 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a40495-9eee-437d-bc4e-fe976c5c5d41 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:46.967030 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a40495-9eee-437d-bc4e-fe976c5c5d41 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:46.967309 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a40495-9eee-437d-bc4e-fe976c5c5d41 None admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:46.968037 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a40495-9eee-437d-bc4e-fe976c5c5d41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:09:46.968341 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a40495-9eee-437d-bc4e-fe976c5c5d41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:46.972665 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a40495-9eee-437d-bc4e-fe976c5c5d41 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:46.981997 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 651/1284] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jun 21 18:09:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:09:46.987882 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd9ee7-5c93-415d-a71f-3b11f6939e59 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:47.104040 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd9ee7-5c93-415d-a71f-3b11f6939e59 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:47.120109 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:47.143507 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cf09ee-9460-4196-95d1-394bc0c5702e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:47.144326 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf09ee-9460-4196-95d1-394bc0c5702e None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:47.144645 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf09ee-9460-4196-95d1-394bc0c5702e None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:47.144969 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf09ee-9460-4196-95d1-394bc0c5702e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:47.153576 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cf09ee-9460-4196-95d1-394bc0c5702e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:47.155053 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cf09ee-9460-4196-95d1-394bc0c5702e None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:47.171098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application [None req-29cf09ee-9460-4196-95d1-394bc0c5702e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:09:47.171098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:09:47.171098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:09:47.171098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:09:47.171098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:09:47.171098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:09:47.171098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:09:47.171098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:09:47.171098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:09:47.171098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:09:47.171098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:09:47.171098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:09:47.171098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:09:47.171098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:09:47.171098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:09:47.180642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:09:47.180642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:09:47.180642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:09:47.180642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:09:47.180642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:09:47.180642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:09:47.180642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:09:47.180642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:09:47.180642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application with Lock( Jun 21 18:09:47.180642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:09:47.180642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:09:47.180642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:09:47.180642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:09:47.180642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:09:47.180642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:09:47.180642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:09:47.180642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:09:47.181667 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:09:47.181667 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:09:47.181667 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:09:47.181667 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:09:47.181667 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:09:47.181667 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:09:47.181667 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:09:47.181667 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: ERROR keystone.server.flask.application Jun 21 18:09:47.181667 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 634/1285] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jun 21 18:09:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:09:47.182210 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a40495-9eee-437d-bc4e-fe976c5c5d41 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:47.237954 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:47.294488 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a40495-9eee-437d-bc4e-fe976c5c5d41 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:47.332941 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7add78-ca82-418b-88d9-c4047bf8a044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:47.333621 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7add78-ca82-418b-88d9-c4047bf8a044 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:47.333883 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7add78-ca82-418b-88d9-c4047bf8a044 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:47.334719 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7add78-ca82-418b-88d9-c4047bf8a044 None admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:47.335531 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7add78-ca82-418b-88d9-c4047bf8a044 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:09:47.335957 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7add78-ca82-418b-88d9-c4047bf8a044 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:47.337443 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7add78-ca82-418b-88d9-c4047bf8a044 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:47.351061 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 652/1286] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jun 21 18:09:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:09:47.357736 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cf09ee-9460-4196-95d1-394bc0c5702e None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:47.464857 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cf09ee-9460-4196-95d1-394bc0c5702e None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:47.500617 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf41612-3a81-43d9-8545-ad6fc241e8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:47.501277 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf41612-3a81-43d9-8545-ad6fc241e8b6 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:47.501561 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf41612-3a81-43d9-8545-ad6fc241e8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:47.501833 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf41612-3a81-43d9-8545-ad6fc241e8b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:47.502658 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf41612-3a81-43d9-8545-ad6fc241e8b6 None admin] RBAC: Adding query filter params (role.id=745ed9065a8143e384ebbd694b29635c, scope.project.id=2a4699d1ec70456da5f8e0aa4b1439b7, user.id=938187967efe49be9c5389d1feef81ed) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:09:47.502967 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf41612-3a81-43d9-8545-ad6fc241e8b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:47.504420 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf41612-3a81-43d9-8545-ad6fc241e8b6 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:47.511460 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 635/1287] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jun 21 18:09:47 2022] GET /identity/v3/role_assignments?user.id=938187967efe49be9c5389d1feef81ed&scope.project.id=2a4699d1ec70456da5f8e0aa4b1439b7&role.id=745ed9065a8143e384ebbd694b29635c => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:09:49.601102 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e57ef6-7ea2-4b22-9c2f-1807bff4281c None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:49.601724 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e57ef6-7ea2-4b22-9c2f-1807bff4281c None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:49.601724 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e57ef6-7ea2-4b22-9c2f-1807bff4281c None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:49.603425 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 653/1288] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:09:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:09:49.608644 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03981c92-208f-4cb9-a866-bf3a3263f314 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:49.609075 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03981c92-208f-4cb9-a866-bf3a3263f314 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:49.609189 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03981c92-208f-4cb9-a866-bf3a3263f314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:49.626779 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-03981c92-208f-4cb9-a866-bf3a3263f314 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:09:49.629813 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-03981c92-208f-4cb9-a866-bf3a3263f314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:09:49.657563 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:49.694276 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 636/1289] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:09:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:09:49.717336 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e57ef6-7ea2-4b22-9c2f-1807bff4281c None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:49.828266 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e57ef6-7ea2-4b22-9c2f-1807bff4281c None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:49.840103 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:49.864026 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b870a9aa-c828-4ee8-8b18-8a8f91402076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:49.864969 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b870a9aa-c828-4ee8-8b18-8a8f91402076 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:49.865257 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b870a9aa-c828-4ee8-8b18-8a8f91402076 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:49.865635 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b870a9aa-c828-4ee8-8b18-8a8f91402076 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:49.866550 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b870a9aa-c828-4ee8-8b18-8a8f91402076 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:49.867961 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b870a9aa-c828-4ee8-8b18-8a8f91402076 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:49.881454 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-b870a9aa-c828-4ee8-8b18-8a8f91402076 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 21 18:09:49.881454 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:09:49.881454 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:09:49.881454 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:09:49.881454 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:09:49.881454 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:09:49.881454 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:09:49.881454 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:09:49.881454 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 21 18:09:49.881454 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:09:49.881454 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:09:49.881454 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:09:49.881454 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:09:49.881454 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:09:49.881454 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:09:49.881454 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:09:49.881454 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:09:49.882708 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:09:49.882708 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:09:49.882708 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:09:49.882708 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:09:49.882708 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:09:49.882708 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:09:49.882708 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:09:49.882708 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:09:49.882708 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:09:49.882708 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:09:49.882708 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:09:49.882708 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:09:49.882708 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:09:49.882708 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:09:49.882708 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:09:49.882708 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:09:49.883842 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:09:49.883842 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:09:49.883842 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:09:49.883842 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:09:49.883842 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:09:49.883842 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:09:49.883842 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:09:49.883842 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:09:49.883842 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:09:49.883842 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:09:49.883842 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:09:49.883842 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:09:49.883842 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 21 18:09:49.883842 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:09:49.883842 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 654/1290] 10.222.0.23 () {56 vars in 1117 bytes} [Tue Jun 21 18:09:49 2022] GET /identity/v3/services/placement => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:09:49.889291 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03981c92-208f-4cb9-a866-bf3a3263f314 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:49.998245 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03981c92-208f-4cb9-a866-bf3a3263f314 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:50.036531 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa2fa35-d988-41f0-91de-b0397c051cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:50.037245 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa2fa35-d988-41f0-91de-b0397c051cc1 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:50.037546 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa2fa35-d988-41f0-91de-b0397c051cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:50.038037 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa2fa35-d988-41f0-91de-b0397c051cc1 None admin] PATH_INFO: `/v3/services` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:50.038946 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa2fa35-d988-41f0-91de-b0397c051cc1 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:09:50.039286 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa2fa35-d988-41f0-91de-b0397c051cc1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:50.040725 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa2fa35-d988-41f0-91de-b0397c051cc1 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:50.046387 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 637/1291] 10.222.0.23 () {56 vars in 1116 bytes} [Tue Jun 21 18:09:49 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:09:50.052375 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b870a9aa-c828-4ee8-8b18-8a8f91402076 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:50.169301 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b870a9aa-c828-4ee8-8b18-8a8f91402076 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:50.203977 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63689e4b-d12a-463b-b00e-5debdfd26abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:50.204964 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63689e4b-d12a-463b-b00e-5debdfd26abf None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:50.205359 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63689e4b-d12a-463b-b00e-5debdfd26abf None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:50.205739 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63689e4b-d12a-463b-b00e-5debdfd26abf None admin] PATH_INFO: `/v3/services` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:50.206653 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63689e4b-d12a-463b-b00e-5debdfd26abf None admin] RBAC: Adding query filter params (type=placement) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:09:50.207032 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63689e4b-d12a-463b-b00e-5debdfd26abf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:50.209106 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63689e4b-d12a-463b-b00e-5debdfd26abf None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:50.214611 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 655/1292] 10.222.0.23 () {56 vars in 1116 bytes} [Tue Jun 21 18:09:50 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:09:52.272695 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99f58c5-337f-45a1-9807-e22187fed2d0 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:52.273254 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99f58c5-337f-45a1-9807-e22187fed2d0 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:52.273254 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99f58c5-337f-45a1-9807-e22187fed2d0 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:52.274615 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 638/1293] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:09:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:09:52.280178 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e33bb-09cb-49ae-922b-9012cef0cd22 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:52.280696 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e33bb-09cb-49ae-922b-9012cef0cd22 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:52.280791 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e33bb-09cb-49ae-922b-9012cef0cd22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:52.299028 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-5c5e33bb-09cb-49ae-922b-9012cef0cd22 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:09:52.300215 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-5c5e33bb-09cb-49ae-922b-9012cef0cd22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:09:52.330736 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:52.369605 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 656/1294] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:09:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:09:52.393034 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99f58c5-337f-45a1-9807-e22187fed2d0 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:52.425201 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:52.507665 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99f58c5-337f-45a1-9807-e22187fed2d0 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:52.545181 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e0242d-ad74-4ac2-be24-2fd6599ddaf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:52.545769 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e0242d-ad74-4ac2-be24-2fd6599ddaf1 None admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:52.546057 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e0242d-ad74-4ac2-be24-2fd6599ddaf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:52.546337 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e0242d-ad74-4ac2-be24-2fd6599ddaf1 None admin] PATH_INFO: `/v3/services` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:52.547089 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e0242d-ad74-4ac2-be24-2fd6599ddaf1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:52.548478 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e0242d-ad74-4ac2-be24-2fd6599ddaf1 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:52.657524 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 639/1295] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Jun 21 18:09:52 2022] POST /identity/v3/services => generated 246 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:09:54.711261 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492b30b7-19a8-4d81-b12b-7a1c010f640d None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:54.711807 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492b30b7-19a8-4d81-b12b-7a1c010f640d None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:54.711807 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492b30b7-19a8-4d81-b12b-7a1c010f640d None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:54.713304 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 657/1296] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:09:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:09:54.718880 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eba705-28f5-48fd-a44f-e9426d7cfef1 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:54.719477 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eba705-28f5-48fd-a44f-e9426d7cfef1 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:54.719477 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eba705-28f5-48fd-a44f-e9426d7cfef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:54.737385 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-41eba705-28f5-48fd-a44f-e9426d7cfef1 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:09:54.738605 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-41eba705-28f5-48fd-a44f-e9426d7cfef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:09:54.816335 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 640/1297] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:09:54 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:09:54.840026 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492b30b7-19a8-4d81-b12b-7a1c010f640d None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:54.951954 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492b30b7-19a8-4d81-b12b-7a1c010f640d None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:54.986685 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1b91e7-d057-4cd7-a636-c7b555fccae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:54.987401 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b91e7-d057-4cd7-a636-c7b555fccae6 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:54.987686 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b91e7-d057-4cd7-a636-c7b555fccae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:54.987962 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b91e7-d057-4cd7-a636-c7b555fccae6 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:54.988649 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1b91e7-d057-4cd7-a636-c7b555fccae6 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:54.990023 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1b91e7-d057-4cd7-a636-c7b555fccae6 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:55.002862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-db1b91e7-d057-4cd7-a636-c7b555fccae6 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 21 18:09:55.002862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:09:55.002862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:09:55.002862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:09:55.002862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:09:55.002862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:09:55.002862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:09:55.002862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:09:55.002862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 21 18:09:55.002862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:09:55.002862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:09:55.002862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:09:55.002862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:09:55.002862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:09:55.002862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:09:55.002862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:09:55.002862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:09:55.007899 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:09:55.007899 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:09:55.007899 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:09:55.007899 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:09:55.007899 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:09:55.007899 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:09:55.007899 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:09:55.007899 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:09:55.007899 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:09:55.007899 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:09:55.007899 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:09:55.007899 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:09:55.007899 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:09:55.007899 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:09:55.007899 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:09:55.007899 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:09:55.009070 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:09:55.009070 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:09:55.009070 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:09:55.009070 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:09:55.009070 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:09:55.009070 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:09:55.009070 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:09:55.009070 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:09:55.009070 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:09:55.009070 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:09:55.009070 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:09:55.009070 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:09:55.009070 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 21 18:09:55.009070 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:09:55.009070 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 658/1298] 10.222.0.23 () {56 vars in 1117 bytes} [Tue Jun 21 18:09:54 2022] GET /identity/v3/services/placement => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:09:55.011256 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eba705-28f5-48fd-a44f-e9426d7cfef1 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:55.029965 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:55.109678 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eba705-28f5-48fd-a44f-e9426d7cfef1 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:55.143740 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f2a60-db95-44ed-8cf7-4cc837b0fe07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:55.144436 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f2a60-db95-44ed-8cf7-4cc837b0fe07 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:55.144730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f2a60-db95-44ed-8cf7-4cc837b0fe07 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:55.145012 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f2a60-db95-44ed-8cf7-4cc837b0fe07 None admin] PATH_INFO: `/v3/services` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:55.145721 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2f2a60-db95-44ed-8cf7-4cc837b0fe07 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:09:55.146032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2f2a60-db95-44ed-8cf7-4cc837b0fe07 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:55.147560 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2f2a60-db95-44ed-8cf7-4cc837b0fe07 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:55.152847 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 641/1299] 10.222.0.23 () {56 vars in 1116 bytes} [Tue Jun 21 18:09:55 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:09:55.158957 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1b91e7-d057-4cd7-a636-c7b555fccae6 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:55.193044 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:55.264311 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1b91e7-d057-4cd7-a636-c7b555fccae6 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:55.303836 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74153b40-9299-435d-87c5-600744a31158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:55.304535 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74153b40-9299-435d-87c5-600744a31158 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:55.304819 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74153b40-9299-435d-87c5-600744a31158 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:55.305099 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74153b40-9299-435d-87c5-600744a31158 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:55.305885 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74153b40-9299-435d-87c5-600744a31158 None admin] RBAC: Adding query filter params (interface=public, service_id=8e99cc6a9d144c70a9ae5c62f3f46ac7, region_id=RegionOne) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:09:55.306197 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74153b40-9299-435d-87c5-600744a31158 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:55.307562 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74153b40-9299-435d-87c5-600744a31158 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:55.312779 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 659/1300] 10.222.0.23 () {56 vars in 1251 bytes} [Tue Jun 21 18:09:55 2022] GET /identity/v3/endpoints?service_id=8e99cc6a9d144c70a9ae5c62f3f46ac7&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:09:55.319309 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f2a60-db95-44ed-8cf7-4cc837b0fe07 None admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:55.403087 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:55.420731 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f2a60-db95-44ed-8cf7-4cc837b0fe07 None admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:55.455554 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44069c92-7649-4921-8abf-d4fc621d0fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:55.456211 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44069c92-7649-4921-8abf-d4fc621d0fd8 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:55.456522 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44069c92-7649-4921-8abf-d4fc621d0fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:55.456797 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44069c92-7649-4921-8abf-d4fc621d0fd8 None admin] PATH_INFO: `/v3/services` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:55.457466 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44069c92-7649-4921-8abf-d4fc621d0fd8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:55.458813 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44069c92-7649-4921-8abf-d4fc621d0fd8 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:55.465901 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 642/1301] 10.222.0.23 () {56 vars in 1087 bytes} [Tue Jun 21 18:09:55 2022] GET /identity/v3/services => generated 2191 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:09:57.544401 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948aa82c-e24d-45f5-a9f9-4f23674c7210 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:57.544972 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948aa82c-e24d-45f5-a9f9-4f23674c7210 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:57.545068 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948aa82c-e24d-45f5-a9f9-4f23674c7210 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:57.546426 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 660/1302] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:09:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:09:57.552828 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c5407-5cd3-4284-82aa-75dbe9b19de2 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:57.553260 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c5407-5cd3-4284-82aa-75dbe9b19de2 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:57.553404 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c5407-5cd3-4284-82aa-75dbe9b19de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:57.570445 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-ae3c5407-5cd3-4284-82aa-75dbe9b19de2 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:09:57.575538 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-ae3c5407-5cd3-4284-82aa-75dbe9b19de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:09:57.638167 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 643/1303] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:09:57 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:09:57.662883 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948aa82c-e24d-45f5-a9f9-4f23674c7210 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:57.759437 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:57.766252 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948aa82c-e24d-45f5-a9f9-4f23674c7210 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:57.802360 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1623a4eb-1512-463a-b17c-bb4a25fab097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:57.802360 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1623a4eb-1512-463a-b17c-bb4a25fab097 None admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:57.802360 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1623a4eb-1512-463a-b17c-bb4a25fab097 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:57.802360 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1623a4eb-1512-463a-b17c-bb4a25fab097 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:57.803306 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1623a4eb-1512-463a-b17c-bb4a25fab097 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:57.803306 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1623a4eb-1512-463a-b17c-bb4a25fab097 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:57.815540 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application [None req-1623a4eb-1512-463a-b17c-bb4a25fab097 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 21 18:09:57.815540 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:09:57.815540 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:09:57.815540 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:09:57.815540 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:09:57.815540 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:09:57.815540 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:09:57.815540 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:09:57.815540 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 21 18:09:57.815540 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:09:57.815540 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:09:57.815540 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:09:57.815540 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:09:57.815540 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:09:57.815540 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:09:57.815540 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:09:57.815540 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:09:57.816853 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:09:57.816853 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:09:57.816853 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:09:57.816853 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:09:57.816853 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:09:57.816853 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:09:57.816853 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:09:57.816853 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:09:57.816853 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:09:57.816853 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:09:57.816853 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:09:57.816853 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:09:57.816853 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:09:57.816853 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:09:57.816853 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:09:57.816853 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:09:57.818107 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:09:57.818107 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application with Lock( Jun 21 18:09:57.818107 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:09:57.818107 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:09:57.818107 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:09:57.818107 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:09:57.818107 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:09:57.818107 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:09:57.818107 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:09:57.818107 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:09:57.818107 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:09:57.818107 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:09:57.818107 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 21 18:09:57.818107 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: ERROR keystone.server.flask.application Jun 21 18:09:57.818107 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 661/1304] 10.222.0.23 () {56 vars in 1117 bytes} [Tue Jun 21 18:09:57 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:09:57.823356 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3c5407-5cd3-4284-82aa-75dbe9b19de2 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:57.902760 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:09:57.927043 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3c5407-5cd3-4284-82aa-75dbe9b19de2 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:57.964788 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d45b4a-119a-4bcd-a023-d98810a71575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:57.965457 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d45b4a-119a-4bcd-a023-d98810a71575 None admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:57.965742 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d45b4a-119a-4bcd-a023-d98810a71575 None admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:57.966018 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d45b4a-119a-4bcd-a023-d98810a71575 None admin] PATH_INFO: `/v3/services` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:57.966729 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d45b4a-119a-4bcd-a023-d98810a71575 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:09:57.967038 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d45b4a-119a-4bcd-a023-d98810a71575 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:57.968482 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d45b4a-119a-4bcd-a023-d98810a71575 None admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:57.973675 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 644/1305] 10.222.0.23 () {56 vars in 1116 bytes} [Tue Jun 21 18:09:57 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:09:57.980374 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1623a4eb-1512-463a-b17c-bb4a25fab097 None admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:09:58.082806 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1623a4eb-1512-463a-b17c-bb4a25fab097 None admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:09:58.117459 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb69a17-602e-4f79-b611-a5c49fbfe909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:09:58.118118 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb69a17-602e-4f79-b611-a5c49fbfe909 None admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:09:58.118400 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb69a17-602e-4f79-b611-a5c49fbfe909 None admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:09:58.118675 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb69a17-602e-4f79-b611-a5c49fbfe909 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:09:58.119427 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb69a17-602e-4f79-b611-a5c49fbfe909 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:09:58.120900 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb69a17-602e-4f79-b611-a5c49fbfe909 None admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:09:58.228974 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 662/1306] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Jun 21 18:09:57 2022] POST /identity/v3/endpoints => generated 331 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:10:08.051161 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=30 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:10:08.223716 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:18:30.289112 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e2681-8198-4dd5-a23d-0d36bd7422ab None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:18:30.289998 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e2681-8198-4dd5-a23d-0d36bd7422ab None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:18:30.290432 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e2681-8198-4dd5-a23d-0d36bd7422ab None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:18:30.297713 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e852008e-f41f-4a43-b937-2cf0b887fb44 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:18:30.298556 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e852008e-f41f-4a43-b937-2cf0b887fb44 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:18:30.298987 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e852008e-f41f-4a43-b937-2cf0b887fb44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:18:30.301446 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 645/1307] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:18:30 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:18:30.459403 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-e852008e-f41f-4a43-b937-2cf0b887fb44 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:18:30.460923 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-e852008e-f41f-4a43-b937-2cf0b887fb44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:18:30.704946 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 663/1308] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:18:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 408 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:18:30.766023 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a8898-316c-4ba2-a9f9-3af39c455b81 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:18:30.766693 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a8898-316c-4ba2-a9f9-3af39c455b81 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:18:30.767180 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a8898-316c-4ba2-a9f9-3af39c455b81 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:18:30.769102 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 646/1309] 10.222.0.23 () {54 vars in 964 bytes} [Tue Jun 21 18:18:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:18:30.775617 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e4274d-0c8f-427c-9bb9-a47b0460af55 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:18:30.776239 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e4274d-0c8f-427c-9bb9-a47b0460af55 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:18:30.776758 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e4274d-0c8f-427c-9bb9-a47b0460af55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:18:31.059254 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-c6e4274d-0c8f-427c-9bb9-a47b0460af55 None None] MFA Rules not processed for user `59c7f07e58ba4db9a7d402d8f3da1e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:18:31.061510 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-c6e4274d-0c8f-427c-9bb9-a47b0460af55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:18:31.087970 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:18:31.167110 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 664/1310] 10.222.0.23 () {58 vars in 1061 bytes} [Tue Jun 21 18:18:30 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 392 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:18:31.168460 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40a8898-316c-4ba2-a9f9-3af39c455b81 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:18:31.170483 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-a40a8898-316c-4ba2-a9f9-3af39c455b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:18:31.332157 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40a8898-316c-4ba2-a9f9-3af39c455b81 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:18:31.422888 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b14d5a-5437-4d1d-8ea8-528f97a4cdc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:18:31.423549 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b14d5a-5437-4d1d-8ea8-528f97a4cdc2 service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:18:31.423829 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b14d5a-5437-4d1d-8ea8-528f97a4cdc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:18:31.424292 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b14d5a-5437-4d1d-8ea8-528f97a4cdc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:18:31.426684 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-91b14d5a-5437-4d1d-8ea8-528f97a4cdc2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:18:31.499168 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:18:31.573083 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b14d5a-5437-4d1d-8ea8-528f97a4cdc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:18:31.574665 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b14d5a-5437-4d1d-8ea8-528f97a4cdc2 service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:18:31.614252 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 647/1311] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:18:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:18:34.031775 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49857604-6a3a-4046-9743-86fd35946296 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:18:34.032327 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49857604-6a3a-4046-9743-86fd35946296 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:18:34.032446 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49857604-6a3a-4046-9743-86fd35946296 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:18:34.033878 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 665/1312] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:18:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:18:34.040264 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14b9b26-3244-4d1c-af9e-da6b57c8b4de None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:18:34.040824 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14b9b26-3244-4d1c-af9e-da6b57c8b4de None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:18:34.040824 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14b9b26-3244-4d1c-af9e-da6b57c8b4de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:18:34.116953 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-f14b9b26-3244-4d1c-af9e-da6b57c8b4de None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:18:34.118163 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-f14b9b26-3244-4d1c-af9e-da6b57c8b4de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:18:34.216159 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 648/1313] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:18:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:18:34.255946 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49857604-6a3a-4046-9743-86fd35946296 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:18:34.257502 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-49857604-6a3a-4046-9743-86fd35946296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:18:34.363836 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49857604-6a3a-4046-9743-86fd35946296 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:18:34.437929 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:18:34.441027 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9945fd-0da4-440d-a76a-7373501f25f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:18:34.442478 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9945fd-0da4-440d-a76a-7373501f25f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:18:34.442846 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9945fd-0da4-440d-a76a-7373501f25f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:18:34.443203 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9945fd-0da4-440d-a76a-7373501f25f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:18:34.445005 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-6f9945fd-0da4-440d-a76a-7373501f25f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:18:34.556562 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9945fd-0da4-440d-a76a-7373501f25f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:18:34.557894 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9945fd-0da4-440d-a76a-7373501f25f4 service neutron] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:18:34.576072 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 666/1314] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:18:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:18:37.287396 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19975c85-bd3b-4bc6-b7af-760db1cebdd6 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:18:37.288003 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19975c85-bd3b-4bc6-b7af-760db1cebdd6 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:18:37.288003 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19975c85-bd3b-4bc6-b7af-760db1cebdd6 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:18:37.289609 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 649/1315] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:18:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:18:37.294982 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552fd68-c24a-400d-8b98-ccc8c978bd37 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:18:37.295420 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552fd68-c24a-400d-8b98-ccc8c978bd37 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:18:37.295524 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552fd68-c24a-400d-8b98-ccc8c978bd37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:18:37.358685 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-2552fd68-c24a-400d-8b98-ccc8c978bd37 None None] MFA Rules not processed for user `623a005b923a4f81bdcab9453639598a`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:18:37.359886 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-2552fd68-c24a-400d-8b98-ccc8c978bd37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:18:37.411278 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:18:37.455931 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 667/1316] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:18:37 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:18:37.495341 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19975c85-bd3b-4bc6-b7af-760db1cebdd6 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:18:37.497041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-19975c85-bd3b-4bc6-b7af-760db1cebdd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:18:37.508051 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:18:37.608261 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19975c85-bd3b-4bc6-b7af-760db1cebdd6 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:18:37.684861 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fc6b87-48db-4a9c-a847-6033a50cb6c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:18:37.685868 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc6b87-48db-4a9c-a847-6033a50cb6c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:18:37.686353 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc6b87-48db-4a9c-a847-6033a50cb6c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:18:37.686667 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc6b87-48db-4a9c-a847-6033a50cb6c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:18:37.688311 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-89fc6b87-48db-4a9c-a847-6033a50cb6c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:18:37.754734 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:18:37.803336 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fc6b87-48db-4a9c-a847-6033a50cb6c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:18:37.805122 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fc6b87-48db-4a9c-a847-6033a50cb6c1 service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:18:37.824818 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 650/1317] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:18:37 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:18:41.035724 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa840667-3aae-43cf-95fd-7646a916aef5 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:18:41.036281 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa840667-3aae-43cf-95fd-7646a916aef5 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:18:41.036281 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa840667-3aae-43cf-95fd-7646a916aef5 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:18:41.037796 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 668/1318] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:18:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:18:41.043075 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bbe1b6-e8f6-472a-82e1-c39170aa5382 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:18:41.043557 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bbe1b6-e8f6-472a-82e1-c39170aa5382 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:18:41.043659 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bbe1b6-e8f6-472a-82e1-c39170aa5382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:18:41.108359 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-61bbe1b6-e8f6-472a-82e1-c39170aa5382 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:18:41.109693 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-61bbe1b6-e8f6-472a-82e1-c39170aa5382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:18:41.205176 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 651/1319] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:18:41 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:18:41.245898 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa840667-3aae-43cf-95fd-7646a916aef5 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:18:41.247431 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-fa840667-3aae-43cf-95fd-7646a916aef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:18:41.351674 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa840667-3aae-43cf-95fd-7646a916aef5 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:18:41.425370 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0c2829-f709-48d4-a654-d6b8705f24e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:18:41.426018 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0c2829-f709-48d4-a654-d6b8705f24e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:18:41.426304 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0c2829-f709-48d4-a654-d6b8705f24e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:18:41.426583 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0c2829-f709-48d4-a654-d6b8705f24e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:18:41.428225 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-9f0c2829-f709-48d4-a654-d6b8705f24e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:18:41.435597 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:18:41.538641 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0c2829-f709-48d4-a654-d6b8705f24e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:18:41.540056 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0c2829-f709-48d4-a654-d6b8705f24e0 service neutron] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:18:41.559211 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 669/1320] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:18:41 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:18:44.931649 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8fdd95-5632-4171-99ed-0e29f6fb7fb7 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:18:44.932238 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8fdd95-5632-4171-99ed-0e29f6fb7fb7 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:18:44.932678 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8fdd95-5632-4171-99ed-0e29f6fb7fb7 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:18:44.934155 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 652/1321] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:18:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:18:44.940233 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14edd68-e5fa-4831-a0cc-0c36106cf623 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:18:44.940776 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14edd68-e5fa-4831-a0cc-0c36106cf623 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:18:44.940954 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14edd68-e5fa-4831-a0cc-0c36106cf623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:18:45.008062 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-f14edd68-e5fa-4831-a0cc-0c36106cf623 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:18:45.009299 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-f14edd68-e5fa-4831-a0cc-0c36106cf623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:18:45.068180 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:18:45.107721 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 670/1322] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:18:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:18:45.145816 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8fdd95-5632-4171-99ed-0e29f6fb7fb7 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:18:45.147571 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-fd8fdd95-5632-4171-99ed-0e29f6fb7fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:18:45.167487 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:18:45.261860 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8fdd95-5632-4171-99ed-0e29f6fb7fb7 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:18:45.345026 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9308f7d3-6a01-4357-b3a3-95a85bc10c78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:18:45.345694 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9308f7d3-6a01-4357-b3a3-95a85bc10c78 service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:18:45.346166 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9308f7d3-6a01-4357-b3a3-95a85bc10c78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:18:45.346502 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9308f7d3-6a01-4357-b3a3-95a85bc10c78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:18:45.348268 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-9308f7d3-6a01-4357-b3a3-95a85bc10c78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:18:45.425692 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:18:45.464486 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9308f7d3-6a01-4357-b3a3-95a85bc10c78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:18:45.465961 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9308f7d3-6a01-4357-b3a3-95a85bc10c78 service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:18:45.484785 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 653/1323] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:18:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:18:47.632346 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c699936-f1f2-42b4-b0b2-c3a0c3ae9d36 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:18:47.632933 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c699936-f1f2-42b4-b0b2-c3a0c3ae9d36 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:18:47.632933 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c699936-f1f2-42b4-b0b2-c3a0c3ae9d36 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:18:47.634483 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 671/1324] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:18:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:18:47.639795 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec5cac0-523b-4e30-a5f2-cafbc3069485 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:18:47.640211 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec5cac0-523b-4e30-a5f2-cafbc3069485 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:18:47.640564 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec5cac0-523b-4e30-a5f2-cafbc3069485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:18:47.705826 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-0ec5cac0-523b-4e30-a5f2-cafbc3069485 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:18:47.707031 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-0ec5cac0-523b-4e30-a5f2-cafbc3069485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:18:47.809507 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 654/1325] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:18:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:18:47.847020 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c699936-f1f2-42b4-b0b2-c3a0c3ae9d36 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:18:47.848623 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-4c699936-f1f2-42b4-b0b2-c3a0c3ae9d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:18:47.959578 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c699936-f1f2-42b4-b0b2-c3a0c3ae9d36 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:18:48.042841 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eea99b-5eca-4e9b-a279-a4e3869f05f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:18:48.043500 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eea99b-5eca-4e9b-a279-a4e3869f05f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:18:48.043785 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eea99b-5eca-4e9b-a279-a4e3869f05f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:18:48.044066 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eea99b-5eca-4e9b-a279-a4e3869f05f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:18:48.045776 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-57eea99b-5eca-4e9b-a279-a4e3869f05f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:18:48.062888 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:18:48.165474 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eea99b-5eca-4e9b-a279-a4e3869f05f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:18:48.167046 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eea99b-5eca-4e9b-a279-a4e3869f05f1 service neutron] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:18:48.188196 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 672/1326] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:18:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:18:50.342679 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d571723a-1666-4d4f-bae2-ebcc4aa68e70 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:18:50.343160 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d571723a-1666-4d4f-bae2-ebcc4aa68e70 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:18:50.343322 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d571723a-1666-4d4f-bae2-ebcc4aa68e70 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:18:50.344731 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 655/1327] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:18:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:18:50.351115 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc0191b-82bc-41db-861e-6fa722ae11f4 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:18:50.351596 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc0191b-82bc-41db-861e-6fa722ae11f4 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:18:50.351692 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc0191b-82bc-41db-861e-6fa722ae11f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:18:50.418171 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-4fc0191b-82bc-41db-861e-6fa722ae11f4 None None] MFA Rules not processed for user `623a005b923a4f81bdcab9453639598a`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:18:50.419441 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-4fc0191b-82bc-41db-861e-6fa722ae11f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:18:50.487271 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:18:50.525396 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 673/1328] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:18:50 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:18:50.564574 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d571723a-1666-4d4f-bae2-ebcc4aa68e70 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:18:50.566125 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-d571723a-1666-4d4f-bae2-ebcc4aa68e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:18:50.601645 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:18:50.676197 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d571723a-1666-4d4f-bae2-ebcc4aa68e70 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:18:50.749907 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a2bb3-0315-452b-997a-e7c5cedb8887 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:18:50.750472 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a2bb3-0315-452b-997a-e7c5cedb8887 service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:18:50.750775 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a2bb3-0315-452b-997a-e7c5cedb8887 service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:18:50.751031 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a2bb3-0315-452b-997a-e7c5cedb8887 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:18:50.752744 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-457a2bb3-0315-452b-997a-e7c5cedb8887 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:18:50.837059 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:18:50.873590 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457a2bb3-0315-452b-997a-e7c5cedb8887 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:18:50.875137 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457a2bb3-0315-452b-997a-e7c5cedb8887 service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:18:50.898314 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 656/1329] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:18:50 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:18:53.590841 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a31ad0-50c6-4295-bbd3-856ca88c8624 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:18:53.591390 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a31ad0-50c6-4295-bbd3-856ca88c8624 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:18:53.591390 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a31ad0-50c6-4295-bbd3-856ca88c8624 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:18:53.592843 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 674/1330] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:18:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:18:53.598828 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2992685a-cd46-4f8e-b2e9-cba711de6e48 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:18:53.599263 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2992685a-cd46-4f8e-b2e9-cba711de6e48 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:18:53.599375 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2992685a-cd46-4f8e-b2e9-cba711de6e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:18:53.664702 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-2992685a-cd46-4f8e-b2e9-cba711de6e48 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:18:53.665939 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-2992685a-cd46-4f8e-b2e9-cba711de6e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:18:53.761659 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 657/1331] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:18:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:18:53.814218 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064f7d01-55f2-4061-982b-b8059103120d None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:18:53.814797 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064f7d01-55f2-4061-982b-b8059103120d None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:18:53.815949 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064f7d01-55f2-4061-982b-b8059103120d None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:18:53.817127 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 675/1332] 10.222.0.23 () {54 vars in 964 bytes} [Tue Jun 21 18:18:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:18:53.823432 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d312a8-040f-4fb4-b13b-a5694e63dcd7 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:18:53.823719 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d312a8-040f-4fb4-b13b-a5694e63dcd7 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:18:53.823999 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d312a8-040f-4fb4-b13b-a5694e63dcd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:18:53.869215 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:18:53.900101 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-18d312a8-040f-4fb4-b13b-a5694e63dcd7 None None] MFA Rules not processed for user `59c7f07e58ba4db9a7d402d8f3da1e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:18:53.901334 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-18d312a8-040f-4fb4-b13b-a5694e63dcd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:18:53.978860 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 658/1333] 10.222.0.23 () {58 vars in 1061 bytes} [Tue Jun 21 18:18:53 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:18:53.988864 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064f7d01-55f2-4061-982b-b8059103120d None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:18:53.990425 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-064f7d01-55f2-4061-982b-b8059103120d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:18:54.095085 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064f7d01-55f2-4061-982b-b8059103120d None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:18:54.166987 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e6da0d-6b88-4596-9c5c-b3fbbf15497a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:18:54.167632 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e6da0d-6b88-4596-9c5c-b3fbbf15497a service neutron] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:18:54.167914 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e6da0d-6b88-4596-9c5c-b3fbbf15497a service neutron] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:18:54.168189 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e6da0d-6b88-4596-9c5c-b3fbbf15497a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:18:54.169856 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-61e6da0d-6b88-4596-9c5c-b3fbbf15497a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:18:54.196290 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:18:54.276220 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e6da0d-6b88-4596-9c5c-b3fbbf15497a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:18:54.277678 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e6da0d-6b88-4596-9c5c-b3fbbf15497a service neutron] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:18:54.295737 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 676/1334] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:18:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:18:56.620121 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc644078-7b6e-4d46-8208-75a89b6de42d None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:18:56.620671 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc644078-7b6e-4d46-8208-75a89b6de42d None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:18:56.620671 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc644078-7b6e-4d46-8208-75a89b6de42d None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:18:56.622055 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 659/1335] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:18:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:18:56.627197 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774b51c5-2899-4abe-8c4c-143783bce7bf None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:18:56.627623 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774b51c5-2899-4abe-8c4c-143783bce7bf None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:18:56.627771 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774b51c5-2899-4abe-8c4c-143783bce7bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:18:56.693583 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-774b51c5-2899-4abe-8c4c-143783bce7bf None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:18:56.694808 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-774b51c5-2899-4abe-8c4c-143783bce7bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:18:56.769917 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:18:56.791966 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 677/1336] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:18:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:18:56.839255 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc644078-7b6e-4d46-8208-75a89b6de42d None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:18:56.841241 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-dc644078-7b6e-4d46-8208-75a89b6de42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:18:56.963770 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc644078-7b6e-4d46-8208-75a89b6de42d None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:18:56.994786 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:18:57.048325 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37def406-5088-421c-8218-5651663830c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:18:57.049018 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37def406-5088-421c-8218-5651663830c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:18:57.049333 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37def406-5088-421c-8218-5651663830c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:18:57.049577 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37def406-5088-421c-8218-5651663830c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:18:57.051270 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-37def406-5088-421c-8218-5651663830c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:18:57.162558 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37def406-5088-421c-8218-5651663830c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:18:57.164021 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37def406-5088-421c-8218-5651663830c2 service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:18:57.182922 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 660/1337] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:18:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:19:00.276712 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16407989-ddd0-4498-8ac7-a58b03564431 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:00.277293 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16407989-ddd0-4498-8ac7-a58b03564431 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:00.277293 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16407989-ddd0-4498-8ac7-a58b03564431 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:00.278624 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 678/1338] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:19:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:19:00.284338 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc690e-7514-48ca-a371-b3f6bcbc94bd None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:00.284817 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc690e-7514-48ca-a371-b3f6bcbc94bd None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:00.284927 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc690e-7514-48ca-a371-b3f6bcbc94bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:00.348153 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-79fc690e-7514-48ca-a371-b3f6bcbc94bd None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:19:00.349387 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-79fc690e-7514-48ca-a371-b3f6bcbc94bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:00.357491 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:19:00.446232 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 661/1339] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:19:00 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:19:00.485553 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16407989-ddd0-4498-8ac7-a58b03564431 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:19:00.489527 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-16407989-ddd0-4498-8ac7-a58b03564431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:00.599786 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16407989-ddd0-4498-8ac7-a58b03564431 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:19:00.612117 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1735e6d1-fc2c-483c-b902-f5dccf71c6be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:19:00.612825 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1735e6d1-fc2c-483c-b902-f5dccf71c6be service neutron] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:00.613158 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1735e6d1-fc2c-483c-b902-f5dccf71c6be service neutron] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:00.613438 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1735e6d1-fc2c-483c-b902-f5dccf71c6be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:00.615340 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-1735e6d1-fc2c-483c-b902-f5dccf71c6be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:00.717439 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:19:00.719696 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1735e6d1-fc2c-483c-b902-f5dccf71c6be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:19:00.721841 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1735e6d1-fc2c-483c-b902-f5dccf71c6be service neutron] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:19:00.741806 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 679/1340] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:19:00 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:19:05.529788 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02abf86-269b-4446-b86d-c5768433c67c None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:05.530279 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02abf86-269b-4446-b86d-c5768433c67c None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:05.530378 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02abf86-269b-4446-b86d-c5768433c67c None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:05.532485 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 662/1341] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:19:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:19:05.538419 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b08227-1e91-4249-ab48-c66aef2659d4 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:05.539509 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b08227-1e91-4249-ab48-c66aef2659d4 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:05.539866 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b08227-1e91-4249-ab48-c66aef2659d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:05.596351 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-a4b08227-1e91-4249-ab48-c66aef2659d4 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:19:05.598371 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-a4b08227-1e91-4249-ab48-c66aef2659d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:05.684512 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 680/1342] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:19:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:19:05.726274 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02abf86-269b-4446-b86d-c5768433c67c None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:19:05.727189 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-b02abf86-269b-4446-b86d-c5768433c67c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:05.753941 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02abf86-269b-4446-b86d-c5768433c67c None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:19:05.757963 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b192c31-bbb2-4c42-a1bc-f83d558399f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:19:05.758678 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b192c31-bbb2-4c42-a1bc-f83d558399f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:05.758915 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b192c31-bbb2-4c42-a1bc-f83d558399f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:05.759194 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b192c31-bbb2-4c42-a1bc-f83d558399f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:05.769969 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b192c31-bbb2-4c42-a1bc-f83d558399f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:19:05.771424 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b192c31-bbb2-4c42-a1bc-f83d558399f5 service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:19:05.784184 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 663/1343] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:19:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:19:09.195396 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5b607-838f-48a8-97f8-67317396ed32 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:09.196239 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5b607-838f-48a8-97f8-67317396ed32 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:09.196239 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5b607-838f-48a8-97f8-67317396ed32 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:09.197514 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 681/1344] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:19:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:19:09.203257 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff18ee85-f95e-4d1c-917d-66ce299521ec None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:09.203851 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff18ee85-f95e-4d1c-917d-66ce299521ec None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:09.203851 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff18ee85-f95e-4d1c-917d-66ce299521ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:09.235516 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-ff18ee85-f95e-4d1c-917d-66ce299521ec None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:19:09.237826 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-ff18ee85-f95e-4d1c-917d-66ce299521ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:09.246844 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 664/1345] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:19:09 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:19:09.286310 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c5b607-838f-48a8-97f8-67317396ed32 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:19:09.297546 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c5b607-838f-48a8-97f8-67317396ed32 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:19:09.301959 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c87b61d-585b-4369-9138-5456a00bdd7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:19:09.302705 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c87b61d-585b-4369-9138-5456a00bdd7b service neutron] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:09.303062 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c87b61d-585b-4369-9138-5456a00bdd7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:09.303409 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c87b61d-585b-4369-9138-5456a00bdd7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:09.312951 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c87b61d-585b-4369-9138-5456a00bdd7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:19:09.314394 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c87b61d-585b-4369-9138-5456a00bdd7b service neutron] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:19:09.324898 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 682/1346] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:19:09 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:19:13.705716 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449be892-0068-475b-968e-db8514719b39 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:13.706273 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449be892-0068-475b-968e-db8514719b39 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:13.706273 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449be892-0068-475b-968e-db8514719b39 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:13.707772 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 665/1347] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:19:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:19:13.713239 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc83e26-132a-4127-b645-3a884338d4d7 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:13.713656 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc83e26-132a-4127-b645-3a884338d4d7 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:13.713764 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc83e26-132a-4127-b645-3a884338d4d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:13.732852 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-7dc83e26-132a-4127-b645-3a884338d4d7 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:19:13.734035 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-7dc83e26-132a-4127-b645-3a884338d4d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:13.743227 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 683/1348] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:19:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:19:13.782482 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449be892-0068-475b-968e-db8514719b39 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:19:13.794013 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449be892-0068-475b-968e-db8514719b39 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:19:13.804479 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec537cb-5099-468a-acb9-ee724a674d0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:19:13.804479 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec537cb-5099-468a-acb9-ee724a674d0a service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:13.804479 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec537cb-5099-468a-acb9-ee724a674d0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:13.804479 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec537cb-5099-468a-acb9-ee724a674d0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:13.809387 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec537cb-5099-468a-acb9-ee724a674d0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:19:13.812114 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec537cb-5099-468a-acb9-ee724a674d0a service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:19:13.823525 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 666/1349] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:19:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:19:18.952501 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a644d-126c-49c2-9dcb-24f102151805 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:18.953363 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a644d-126c-49c2-9dcb-24f102151805 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:18.953804 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a644d-126c-49c2-9dcb-24f102151805 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:18.960809 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 684/1350] 10.222.0.23 () {54 vars in 921 bytes} [Tue Jun 21 18:19:18 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:19:18.966161 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb7e31-c74a-495e-85c3-1d0cacc9b6c1 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:18.967006 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb7e31-c74a-495e-85c3-1d0cacc9b6c1 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:18.967516 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb7e31-c74a-495e-85c3-1d0cacc9b6c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:19.046045 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-ffdb7e31-c74a-495e-85c3-1d0cacc9b6c1 None None] MFA Rules not processed for user `938187967efe49be9c5389d1feef81ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:19:19.047275 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-ffdb7e31-c74a-495e-85c3-1d0cacc9b6c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:19.097415 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:19:19.139037 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 667/1351] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Jun 21 18:19:18 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:19:23.466946 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d2c42c-b037-4e0c-84d4-c47a45b11ba3 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:23.467864 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d2c42c-b037-4e0c-84d4-c47a45b11ba3 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:23.468133 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d2c42c-b037-4e0c-84d4-c47a45b11ba3 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:23.469895 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 685/1352] 10.222.0.23 () {54 vars in 921 bytes} [Tue Jun 21 18:19:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:19:23.476125 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d3250c-e9e2-4ffd-abaf-0fe0ab98cb8f None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:23.476626 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d3250c-e9e2-4ffd-abaf-0fe0ab98cb8f None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:23.476999 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d3250c-e9e2-4ffd-abaf-0fe0ab98cb8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:23.496334 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-81d3250c-e9e2-4ffd-abaf-0fe0ab98cb8f None None] MFA Rules not processed for user `938187967efe49be9c5389d1feef81ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:19:23.497726 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-81d3250c-e9e2-4ffd-abaf-0fe0ab98cb8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:23.507051 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 668/1353] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Jun 21 18:19:23 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:19:23.728582 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=60 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:19:25.323320 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63426bf7-baa2-4f3a-8f7a-550628536244 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:25.324314 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63426bf7-baa2-4f3a-8f7a-550628536244 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:25.324619 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63426bf7-baa2-4f3a-8f7a-550628536244 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:25.326018 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 686/1354] 10.222.0.23 () {54 vars in 921 bytes} [Tue Jun 21 18:19:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:19:25.331152 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e795b336-bb2e-4ceb-bc0f-74eb9f548c81 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:25.331583 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e795b336-bb2e-4ceb-bc0f-74eb9f548c81 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:25.331751 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e795b336-bb2e-4ceb-bc0f-74eb9f548c81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:25.350547 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-e795b336-bb2e-4ceb-bc0f-74eb9f548c81 None None] MFA Rules not processed for user `938187967efe49be9c5389d1feef81ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:19:25.351746 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-e795b336-bb2e-4ceb-bc0f-74eb9f548c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:25.360414 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 669/1355] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Jun 21 18:19:25 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:19:28.879277 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afab1c38-55eb-4bc8-96cd-4c50485cb543 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:28.879812 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afab1c38-55eb-4bc8-96cd-4c50485cb543 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:28.879812 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afab1c38-55eb-4bc8-96cd-4c50485cb543 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:28.881359 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 687/1356] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:19:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:19:28.890187 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2233b-f8c9-44bd-b7bf-bc813e69c3ad None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:28.890622 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2233b-f8c9-44bd-b7bf-bc813e69c3ad None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:28.890813 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2233b-f8c9-44bd-b7bf-bc813e69c3ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:28.913707 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-7be2233b-f8c9-44bd-b7bf-bc813e69c3ad None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:19:28.915662 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-7be2233b-f8c9-44bd-b7bf-bc813e69c3ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:28.925694 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 670/1357] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:19:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:19:28.991765 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4422621-1e34-4727-af5a-4b15d20b9a0d None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:28.992359 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4422621-1e34-4727-af5a-4b15d20b9a0d None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:28.992912 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4422621-1e34-4727-af5a-4b15d20b9a0d None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:28.994825 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 688/1358] 10.222.0.23 () {54 vars in 961 bytes} [Tue Jun 21 18:19:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:19:29.006955 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56779239-c450-43f6-864c-748ffe4f571a None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:29.009486 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56779239-c450-43f6-864c-748ffe4f571a None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:29.010001 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56779239-c450-43f6-864c-748ffe4f571a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:29.077015 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-56779239-c450-43f6-864c-748ffe4f571a None None] MFA Rules not processed for user `904255dd3c1f406eb10f6fc65f9cd098`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:19:29.078577 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-56779239-c450-43f6-864c-748ffe4f571a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:29.186181 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 671/1359] 10.222.0.23 () {58 vars in 1058 bytes} [Tue Jun 21 18:19:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:19:29.195298 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4422621-1e34-4727-af5a-4b15d20b9a0d None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:19:29.214104 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4422621-1e34-4727-af5a-4b15d20b9a0d None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:19:29.220158 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c89bb0-93e4-4c17-a350-627b48728f31 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904255dd3c1f406eb10f6fc65f9cd098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:19:29.221756 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c89bb0-93e4-4c17-a350-627b48728f31 service nova] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:29.222211 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c89bb0-93e4-4c17-a350-627b48728f31 service nova] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:29.222796 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c89bb0-93e4-4c17-a350-627b48728f31 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:29.232425 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c89bb0-93e4-4c17-a350-627b48728f31 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:19:29.234400 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c89bb0-93e4-4c17-a350-627b48728f31 service nova] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:19:29.245670 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 689/1360] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:19:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:19:31.536878 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedf3c88-df39-463f-b4cf-394946815475 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:31.537421 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedf3c88-df39-463f-b4cf-394946815475 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:31.537421 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedf3c88-df39-463f-b4cf-394946815475 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:31.539295 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 672/1361] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:19:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:19:31.545066 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba8d40-9ac8-4078-8966-3b4500a88cdd None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:31.545524 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba8d40-9ac8-4078-8966-3b4500a88cdd None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:31.545615 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba8d40-9ac8-4078-8966-3b4500a88cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:31.565231 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-2cba8d40-9ac8-4078-8966-3b4500a88cdd None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:19:31.566491 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-2cba8d40-9ac8-4078-8966-3b4500a88cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:31.574707 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 690/1362] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:19:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:19:31.620057 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedf3c88-df39-463f-b4cf-394946815475 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:19:31.635859 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedf3c88-df39-463f-b4cf-394946815475 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:19:31.641361 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d76d9df-f611-4d30-8b34-b8ec5624c784 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904255dd3c1f406eb10f6fc65f9cd098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:19:31.642018 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d76d9df-f611-4d30-8b34-b8ec5624c784 service nova] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:31.642893 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d76d9df-f611-4d30-8b34-b8ec5624c784 service nova] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:31.643178 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d76d9df-f611-4d30-8b34-b8ec5624c784 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:31.652245 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d76d9df-f611-4d30-8b34-b8ec5624c784 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:19:31.653876 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d76d9df-f611-4d30-8b34-b8ec5624c784 service nova] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:19:31.663834 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 673/1363] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:19:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:19:34.302861 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b486c29d-fd9c-4811-ae7e-493d472e430e None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:34.303362 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b486c29d-fd9c-4811-ae7e-493d472e430e None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:34.303362 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b486c29d-fd9c-4811-ae7e-493d472e430e None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:34.305278 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 691/1364] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:19:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:19:34.314971 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21caa1de-aa88-4d0a-ae28-a095a92c27a6 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:34.315900 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21caa1de-aa88-4d0a-ae28-a095a92c27a6 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:34.316450 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21caa1de-aa88-4d0a-ae28-a095a92c27a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:34.337849 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-21caa1de-aa88-4d0a-ae28-a095a92c27a6 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:19:34.339092 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-21caa1de-aa88-4d0a-ae28-a095a92c27a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:34.348141 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 674/1365] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:19:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:19:34.398610 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b15c1-3924-4c22-a1ad-8676680cb5e3 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:34.398991 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b15c1-3924-4c22-a1ad-8676680cb5e3 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:34.399343 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b15c1-3924-4c22-a1ad-8676680cb5e3 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:34.400862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 692/1366] 10.222.0.23 () {54 vars in 961 bytes} [Tue Jun 21 18:19:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:19:34.406921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd12e5-d6e0-4dfc-9649-84982d451d3e None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:34.407210 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd12e5-d6e0-4dfc-9649-84982d451d3e None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:34.407492 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd12e5-d6e0-4dfc-9649-84982d451d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:34.426778 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-2cbd12e5-d6e0-4dfc-9649-84982d451d3e None None] MFA Rules not processed for user `904255dd3c1f406eb10f6fc65f9cd098`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:19:34.427987 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-2cbd12e5-d6e0-4dfc-9649-84982d451d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:34.437762 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 675/1367] 10.222.0.23 () {58 vars in 1058 bytes} [Tue Jun 21 18:19:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:19:34.446539 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306b15c1-3924-4c22-a1ad-8676680cb5e3 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:19:34.461362 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306b15c1-3924-4c22-a1ad-8676680cb5e3 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:19:34.466918 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c7a2de-5a2c-4e89-a5b6-4abafab6254c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904255dd3c1f406eb10f6fc65f9cd098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:19:34.467572 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7a2de-5a2c-4e89-a5b6-4abafab6254c service nova] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:34.467857 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7a2de-5a2c-4e89-a5b6-4abafab6254c service nova] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:34.468136 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7a2de-5a2c-4e89-a5b6-4abafab6254c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:34.477345 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c7a2de-5a2c-4e89-a5b6-4abafab6254c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:19:34.478951 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c7a2de-5a2c-4e89-a5b6-4abafab6254c service nova] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:19:34.490395 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 693/1368] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:19:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:19:36.849187 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c0b7b6-3c58-4d28-8800-49c5a421964b None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:36.850113 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c0b7b6-3c58-4d28-8800-49c5a421964b None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:36.850396 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c0b7b6-3c58-4d28-8800-49c5a421964b None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:36.852261 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 676/1369] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:19:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:19:36.858006 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52d236-884f-4ffd-a854-cab773285ccb None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:36.858532 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52d236-884f-4ffd-a854-cab773285ccb None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:36.860864 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52d236-884f-4ffd-a854-cab773285ccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:36.881995 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-cb52d236-884f-4ffd-a854-cab773285ccb None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:19:36.883531 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-cb52d236-884f-4ffd-a854-cab773285ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:36.893646 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 694/1370] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:19:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:19:36.939810 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c0b7b6-3c58-4d28-8800-49c5a421964b None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:19:36.959734 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c0b7b6-3c58-4d28-8800-49c5a421964b None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:19:36.965752 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d77166-db74-4ec3-9154-0af76530739d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904255dd3c1f406eb10f6fc65f9cd098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:19:36.966523 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d77166-db74-4ec3-9154-0af76530739d service nova] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:36.966865 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d77166-db74-4ec3-9154-0af76530739d service nova] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:36.967236 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d77166-db74-4ec3-9154-0af76530739d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:36.976758 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d77166-db74-4ec3-9154-0af76530739d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:19:36.978599 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d77166-db74-4ec3-9154-0af76530739d service nova] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:19:36.990453 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 677/1371] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:19:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:19:37.433515 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50bceb0-3a1d-44e1-a569-cfd2e21db67e None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:37.433515 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50bceb0-3a1d-44e1-a569-cfd2e21db67e None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:37.433515 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50bceb0-3a1d-44e1-a569-cfd2e21db67e None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:37.433515 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 695/1372] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:19:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:19:37.434089 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c21e6-3ae7-43a5-a458-f1e592580c85 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:37.434089 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c21e6-3ae7-43a5-a458-f1e592580c85 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:37.434089 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c21e6-3ae7-43a5-a458-f1e592580c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:37.441706 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-150c21e6-3ae7-43a5-a458-f1e592580c85 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:19:37.443029 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-150c21e6-3ae7-43a5-a458-f1e592580c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:37.452414 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 678/1373] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:19:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:19:37.813554 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a5f5f4-c99a-4664-86da-d0bce881d4df None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:37.813554 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a5f5f4-c99a-4664-86da-d0bce881d4df None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:37.813554 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a5f5f4-c99a-4664-86da-d0bce881d4df None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:37.817316 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 696/1374] 10.222.0.23 () {54 vars in 963 bytes} [Tue Jun 21 18:19:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:19:37.821061 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f13e79-7935-4e0f-b3f9-0eac0aff19e0 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:37.826199 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f13e79-7935-4e0f-b3f9-0eac0aff19e0 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:37.826199 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f13e79-7935-4e0f-b3f9-0eac0aff19e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:37.849776 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:19:37.886707 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-d1f13e79-7935-4e0f-b3f9-0eac0aff19e0 None None] MFA Rules not processed for user `77c6e43555944902a22ba2fe17a8e99f`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:19:37.888133 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-d1f13e79-7935-4e0f-b3f9-0eac0aff19e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:37.965535 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 679/1375] 10.222.0.23 () {58 vars in 1060 bytes} [Tue Jun 21 18:19:37 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:19:37.974388 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a5f5f4-c99a-4664-86da-d0bce881d4df None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:19:37.987570 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a5f5f4-c99a-4664-86da-d0bce881d4df None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:19:37.991540 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcab014-319b-4c6b-a060-c8ca6b17ca6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:19:37.992202 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcab014-319b-4c6b-a060-c8ca6b17ca6f service cinder] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:37.992605 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcab014-319b-4c6b-a060-c8ca6b17ca6f service cinder] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:37.992886 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcab014-319b-4c6b-a060-c8ca6b17ca6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:38.003217 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcab014-319b-4c6b-a060-c8ca6b17ca6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:19:38.004729 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcab014-319b-4c6b-a060-c8ca6b17ca6f service cinder] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:19:38.015395 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 697/1376] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:19:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:19:39.554045 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e1bce-9e31-4f9f-b75a-8f1ce4ce5002 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:39.554629 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e1bce-9e31-4f9f-b75a-8f1ce4ce5002 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:39.554629 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e1bce-9e31-4f9f-b75a-8f1ce4ce5002 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:39.555957 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 680/1377] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:19:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:19:39.561174 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218ef9e9-b518-493f-9875-ad17dcd4cccd None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:39.561619 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218ef9e9-b518-493f-9875-ad17dcd4cccd None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:39.561715 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218ef9e9-b518-493f-9875-ad17dcd4cccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:39.580128 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-218ef9e9-b518-493f-9875-ad17dcd4cccd None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:19:39.581458 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-218ef9e9-b518-493f-9875-ad17dcd4cccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:39.589786 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 698/1378] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:19:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:19:39.636546 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237e1bce-9e31-4f9f-b75a-8f1ce4ce5002 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:19:39.652177 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237e1bce-9e31-4f9f-b75a-8f1ce4ce5002 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:19:39.658721 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4e7a14-6de1-4653-9eb2-8bbaa6361cb3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904255dd3c1f406eb10f6fc65f9cd098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:19:39.661675 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e7a14-6de1-4653-9eb2-8bbaa6361cb3 service nova] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:39.661675 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e7a14-6de1-4653-9eb2-8bbaa6361cb3 service nova] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:39.661675 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e7a14-6de1-4653-9eb2-8bbaa6361cb3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:39.668533 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4e7a14-6de1-4653-9eb2-8bbaa6361cb3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:19:39.670214 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4e7a14-6de1-4653-9eb2-8bbaa6361cb3 service nova] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:19:39.680017 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 681/1379] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:19:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:19:41.979323 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850502e-42e3-4b26-8281-9763c0487998 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:41.979830 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850502e-42e3-4b26-8281-9763c0487998 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:41.979830 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850502e-42e3-4b26-8281-9763c0487998 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:41.981174 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 699/1380] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:19:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:19:41.987253 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58dfb44-aa1a-4b60-b4f2-0e15bed0927b None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:41.987850 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58dfb44-aa1a-4b60-b4f2-0e15bed0927b None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:41.987850 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58dfb44-aa1a-4b60-b4f2-0e15bed0927b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:42.006776 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-e58dfb44-aa1a-4b60-b4f2-0e15bed0927b None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:19:42.007977 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-e58dfb44-aa1a-4b60-b4f2-0e15bed0927b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:42.016241 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 682/1381] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:19:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:19:42.062608 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850502e-42e3-4b26-8281-9763c0487998 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:19:42.081082 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850502e-42e3-4b26-8281-9763c0487998 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:19:42.086884 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4450c44-f8fd-4e8a-8e30-553f21acb674 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904255dd3c1f406eb10f6fc65f9cd098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:19:42.087635 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4450c44-f8fd-4e8a-8e30-553f21acb674 service nova] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:42.088283 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4450c44-f8fd-4e8a-8e30-553f21acb674 service nova] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:42.088699 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4450c44-f8fd-4e8a-8e30-553f21acb674 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:42.100250 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4450c44-f8fd-4e8a-8e30-553f21acb674 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:19:42.102062 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4450c44-f8fd-4e8a-8e30-553f21acb674 service nova] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:19:42.114002 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 700/1382] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:19:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:19:44.448543 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf748e6-00ec-44b9-963f-2a65451c43c3 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:44.449784 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf748e6-00ec-44b9-963f-2a65451c43c3 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:44.450376 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf748e6-00ec-44b9-963f-2a65451c43c3 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:44.452017 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 683/1383] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:19:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:19:44.458249 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5543970a-bf78-430b-b81c-b7dcc52a0269 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:44.458249 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5543970a-bf78-430b-b81c-b7dcc52a0269 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:44.458844 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5543970a-bf78-430b-b81c-b7dcc52a0269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:44.477460 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-5543970a-bf78-430b-b81c-b7dcc52a0269 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:19:44.478819 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-5543970a-bf78-430b-b81c-b7dcc52a0269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:44.488480 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 701/1384] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:19:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:19:44.538020 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf748e6-00ec-44b9-963f-2a65451c43c3 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:19:44.555220 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf748e6-00ec-44b9-963f-2a65451c43c3 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:19:44.561112 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eca0438-ff58-40a6-988d-63557249044b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904255dd3c1f406eb10f6fc65f9cd098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:19:44.561767 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eca0438-ff58-40a6-988d-63557249044b service nova] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:44.562048 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eca0438-ff58-40a6-988d-63557249044b service nova] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:44.562320 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eca0438-ff58-40a6-988d-63557249044b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:44.575325 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eca0438-ff58-40a6-988d-63557249044b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:19:44.576686 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eca0438-ff58-40a6-988d-63557249044b service nova] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:19:44.587870 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 684/1385] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:19:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:19:46.871574 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0954bf8-b4ca-43a3-b877-0000d9663eff None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:46.872131 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0954bf8-b4ca-43a3-b877-0000d9663eff None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:46.872236 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0954bf8-b4ca-43a3-b877-0000d9663eff None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:46.874533 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 702/1386] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:19:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:19:46.880515 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad51f9-f373-4ed1-9fba-270212963ae6 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:46.881284 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad51f9-f373-4ed1-9fba-270212963ae6 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:46.881284 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad51f9-f373-4ed1-9fba-270212963ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:46.900442 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-64ad51f9-f373-4ed1-9fba-270212963ae6 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:19:46.901674 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-64ad51f9-f373-4ed1-9fba-270212963ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:46.910588 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 685/1387] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:19:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:19:46.958843 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0954bf8-b4ca-43a3-b877-0000d9663eff None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:19:46.977539 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0954bf8-b4ca-43a3-b877-0000d9663eff None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:19:46.983306 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0108ca17-c7ff-4ba7-b3d6-5512702a6c23 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904255dd3c1f406eb10f6fc65f9cd098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:19:46.983968 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0108ca17-c7ff-4ba7-b3d6-5512702a6c23 service nova] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:46.984253 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0108ca17-c7ff-4ba7-b3d6-5512702a6c23 service nova] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:46.984557 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0108ca17-c7ff-4ba7-b3d6-5512702a6c23 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:46.994605 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0108ca17-c7ff-4ba7-b3d6-5512702a6c23 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:19:46.996349 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0108ca17-c7ff-4ba7-b3d6-5512702a6c23 service nova] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:19:47.006968 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 703/1388] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:19:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:19:49.278511 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992334e8-5652-49bb-9923-3407c2f5d8cc None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:49.279126 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992334e8-5652-49bb-9923-3407c2f5d8cc None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:49.279126 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992334e8-5652-49bb-9923-3407c2f5d8cc None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:49.280482 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 686/1389] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:19:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:19:49.287073 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b800c-ec5e-4ff1-82f9-6ccd6411cb40 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:49.287671 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b800c-ec5e-4ff1-82f9-6ccd6411cb40 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:49.287671 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b800c-ec5e-4ff1-82f9-6ccd6411cb40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:49.307261 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-be5b800c-ec5e-4ff1-82f9-6ccd6411cb40 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:19:49.308537 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-be5b800c-ec5e-4ff1-82f9-6ccd6411cb40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:49.322181 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 704/1390] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:19:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:19:49.370802 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992334e8-5652-49bb-9923-3407c2f5d8cc None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:19:49.385770 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992334e8-5652-49bb-9923-3407c2f5d8cc None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:19:49.392619 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abfcc17-17d4-401e-96a6-e23c0448a769 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904255dd3c1f406eb10f6fc65f9cd098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:19:49.393487 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abfcc17-17d4-401e-96a6-e23c0448a769 service nova] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:49.393892 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abfcc17-17d4-401e-96a6-e23c0448a769 service nova] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:49.394268 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abfcc17-17d4-401e-96a6-e23c0448a769 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:49.404012 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abfcc17-17d4-401e-96a6-e23c0448a769 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:19:49.406918 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abfcc17-17d4-401e-96a6-e23c0448a769 service nova] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:19:49.426680 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 687/1391] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:19:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:19:51.882449 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f832e93-3f7a-478c-8cdd-082e718efd4e None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:51.883667 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f832e93-3f7a-478c-8cdd-082e718efd4e None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:51.884161 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f832e93-3f7a-478c-8cdd-082e718efd4e None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:51.886870 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 705/1392] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:19:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:19:51.892933 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3ee90-8268-4b0c-9d8f-79f451992f3f None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:51.893758 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3ee90-8268-4b0c-9d8f-79f451992f3f None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:51.893758 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3ee90-8268-4b0c-9d8f-79f451992f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:51.913398 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-7cb3ee90-8268-4b0c-9d8f-79f451992f3f None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:19:51.914680 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-7cb3ee90-8268-4b0c-9d8f-79f451992f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:51.922998 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 688/1393] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:19:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:19:51.974133 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f832e93-3f7a-478c-8cdd-082e718efd4e None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:19:51.996730 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f832e93-3f7a-478c-8cdd-082e718efd4e None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:19:52.002768 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96085c0-9e54-43ef-919b-dcbb41c4767a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904255dd3c1f406eb10f6fc65f9cd098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:19:52.003406 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96085c0-9e54-43ef-919b-dcbb41c4767a service nova] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:52.003712 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96085c0-9e54-43ef-919b-dcbb41c4767a service nova] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:52.003997 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96085c0-9e54-43ef-919b-dcbb41c4767a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:52.013881 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96085c0-9e54-43ef-919b-dcbb41c4767a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:19:52.015556 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96085c0-9e54-43ef-919b-dcbb41c4767a service nova] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:19:52.026327 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 706/1394] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:19:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:19:54.158013 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b1d43-4da4-45f9-9360-6155d51068b4 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:54.158563 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b1d43-4da4-45f9-9360-6155d51068b4 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:54.158700 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b1d43-4da4-45f9-9360-6155d51068b4 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:54.160086 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 689/1395] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:19:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:19:54.165717 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8711a51-febb-42bd-9328-7f837325b361 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:54.166185 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8711a51-febb-42bd-9328-7f837325b361 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:54.166284 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8711a51-febb-42bd-9328-7f837325b361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:54.172553 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=52 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:19:54.185394 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-d8711a51-febb-42bd-9328-7f837325b361 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:19:54.192142 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-d8711a51-febb-42bd-9328-7f837325b361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:54.195913 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 707/1396] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:19:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:19:54.260195 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf946af0-28ec-4e29-87e8-10baa204d637 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:54.260518 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf946af0-28ec-4e29-87e8-10baa204d637 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:54.260797 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf946af0-28ec-4e29-87e8-10baa204d637 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:54.262176 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 690/1397] 10.222.0.23 () {54 vars in 962 bytes} [Tue Jun 21 18:19:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:19:54.265053 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c079818-814b-4ef6-a347-68b19063412a None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:54.265351 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c079818-814b-4ef6-a347-68b19063412a None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:54.265827 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c079818-814b-4ef6-a347-68b19063412a None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:54.267621 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644cf7fa-ca36-4c5f-99fd-1c21b4505f65 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:54.267908 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644cf7fa-ca36-4c5f-99fd-1c21b4505f65 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:54.268184 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644cf7fa-ca36-4c5f-99fd-1c21b4505f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:54.269300 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 708/1398] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:19:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:19:54.274619 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d837c9-9dc2-489f-b0a1-f4cb2c46c367 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:54.274913 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d837c9-9dc2-489f-b0a1-f4cb2c46c367 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:54.275193 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d837c9-9dc2-489f-b0a1-f4cb2c46c367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:54.293729 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-a9d837c9-9dc2-489f-b0a1-f4cb2c46c367 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:19:54.296075 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-a9d837c9-9dc2-489f-b0a1-f4cb2c46c367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:54.305249 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 709/1399] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:19:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:19:54.320126 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-644cf7fa-ca36-4c5f-99fd-1c21b4505f65 None None] MFA Rules not processed for user `9672794bc1ca4dcd8259d51d9290384b`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:19:54.321354 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-644cf7fa-ca36-4c5f-99fd-1c21b4505f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:54.358389 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d837c9-9dc2-489f-b0a1-f4cb2c46c367 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:19:54.372802 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d837c9-9dc2-489f-b0a1-f4cb2c46c367 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:19:54.379202 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0548daf-7106-4b5a-8198-2d46f542dbf3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904255dd3c1f406eb10f6fc65f9cd098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:19:54.379897 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0548daf-7106-4b5a-8198-2d46f542dbf3 service nova] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:54.380184 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0548daf-7106-4b5a-8198-2d46f542dbf3 service nova] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:54.380495 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0548daf-7106-4b5a-8198-2d46f542dbf3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:54.384834 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 691/1400] 10.222.0.23 () {58 vars in 1059 bytes} [Tue Jun 21 18:19:54 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:19:54.389265 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0548daf-7106-4b5a-8198-2d46f542dbf3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:19:54.390861 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0548daf-7106-4b5a-8198-2d46f542dbf3 service nova] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:19:54.393840 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644cf7fa-ca36-4c5f-99fd-1c21b4505f65 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:19:54.401446 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644cf7fa-ca36-4c5f-99fd-1c21b4505f65 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:19:54.402118 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 710/1401] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:19:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:19:54.405276 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9006ef-6fe0-462b-bae1-37a704d17e92 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9672794bc1ca4dcd8259d51d9290384b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:19:54.405924 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9006ef-6fe0-462b-bae1-37a704d17e92 service glance] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:54.406211 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9006ef-6fe0-462b-bae1-37a704d17e92 service glance] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:54.406509 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9006ef-6fe0-462b-bae1-37a704d17e92 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:54.414499 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:19:54.416789 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9006ef-6fe0-462b-bae1-37a704d17e92 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:19:54.418292 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9006ef-6fe0-462b-bae1-37a704d17e92 service glance] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:19:54.428203 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 692/1402] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:19:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:19:55.437880 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6137d-7f41-4437-b4be-463ad8238b36 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:55.438337 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6137d-7f41-4437-b4be-463ad8238b36 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:55.439801 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6137d-7f41-4437-b4be-463ad8238b36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:55.489183 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-2ae6137d-7f41-4437-b4be-463ad8238b36 None None] MFA Rules not processed for user `fcfcd9f134e74b09bfdac337c81c8a07`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:19:55.495447 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-2ae6137d-7f41-4437-b4be-463ad8238b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:55.558758 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 711/1403] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Jun 21 18:19:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:19:55.605879 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57c1098-05b3-4985-bab6-b9140d0dfaf3 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:55.606355 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57c1098-05b3-4985-bab6-b9140d0dfaf3 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:55.606559 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57c1098-05b3-4985-bab6-b9140d0dfaf3 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:55.608036 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 693/1404] 10.222.0.23 () {54 vars in 942 bytes} [Tue Jun 21 18:19:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:19:55.615480 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ff06b4-c16d-45d9-a9ca-3b023169f832 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:55.615774 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ff06b4-c16d-45d9-a9ca-3b023169f832 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:55.616058 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ff06b4-c16d-45d9-a9ca-3b023169f832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:55.666623 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-58ff06b4-c16d-45d9-a9ca-3b023169f832 None None] MFA Rules not processed for user `0e0d25565f1543d9a7577288fc09119f`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:19:55.667824 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-58ff06b4-c16d-45d9-a9ca-3b023169f832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:55.743785 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 712/1405] 10.222.0.23 () {58 vars in 1039 bytes} [Tue Jun 21 18:19:55 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:19:55.753718 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57c1098-05b3-4985-bab6-b9140d0dfaf3 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:19:55.767354 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57c1098-05b3-4985-bab6-b9140d0dfaf3 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:19:55.773179 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f06a1ba-b9ad-4a7c-9476-8fe8e94276d8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e0d25565f1543d9a7577288fc09119f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:19:55.773775 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f06a1ba-b9ad-4a7c-9476-8fe8e94276d8 service swift] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:55.774083 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f06a1ba-b9ad-4a7c-9476-8fe8e94276d8 service swift] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:55.774347 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f06a1ba-b9ad-4a7c-9476-8fe8e94276d8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:55.784670 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f06a1ba-b9ad-4a7c-9476-8fe8e94276d8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:19:55.786269 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f06a1ba-b9ad-4a7c-9476-8fe8e94276d8 service swift] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:19:55.795010 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 694/1406] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Jun 21 18:19:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 18:19:56.668744 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c1e7b8-9777-4f60-b37d-f143a2166201 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:56.669891 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c1e7b8-9777-4f60-b37d-f143a2166201 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:56.670499 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c1e7b8-9777-4f60-b37d-f143a2166201 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:56.671972 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 713/1407] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:19:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:19:56.678454 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaaef8b-896e-4275-85cc-90100d90818e None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:56.678990 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaaef8b-896e-4275-85cc-90100d90818e None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:56.678990 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaaef8b-896e-4275-85cc-90100d90818e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:56.700359 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-aaaaef8b-896e-4275-85cc-90100d90818e None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:19:56.701310 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-aaaaef8b-896e-4275-85cc-90100d90818e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:19:56.711173 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 695/1408] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:19:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:19:56.762598 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c1e7b8-9777-4f60-b37d-f143a2166201 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:19:56.783475 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c1e7b8-9777-4f60-b37d-f143a2166201 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:19:56.792219 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2015242-f141-4882-9028-51d2cf8e50a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904255dd3c1f406eb10f6fc65f9cd098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:19:56.793162 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2015242-f141-4882-9028-51d2cf8e50a1 service nova] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:19:56.793519 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2015242-f141-4882-9028-51d2cf8e50a1 service nova] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:19:56.793798 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2015242-f141-4882-9028-51d2cf8e50a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:19:56.804192 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2015242-f141-4882-9028-51d2cf8e50a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:19:56.805817 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2015242-f141-4882-9028-51d2cf8e50a1 service nova] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:19:56.810129 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:19:56.820481 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 714/1409] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:19:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:20:06.694775 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=11 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:20:06.842482 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:20:19.671775 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc7fd9-ccf9-4570-b13a-0242a306b809 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:19.672511 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc7fd9-ccf9-4570-b13a-0242a306b809 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:19.672511 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc7fd9-ccf9-4570-b13a-0242a306b809 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:19.675053 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 696/1410] 10.222.0.23 () {54 vars in 918 bytes} [Tue Jun 21 18:20:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:20:19.681009 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f86e974-a452-4108-b0b5-e120a4ca2d8d None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:19.681572 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f86e974-a452-4108-b0b5-e120a4ca2d8d None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:19.681572 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f86e974-a452-4108-b0b5-e120a4ca2d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:19.701586 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-5f86e974-a452-4108-b0b5-e120a4ca2d8d None None] MFA Rules not processed for user `938187967efe49be9c5389d1feef81ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:20:19.702823 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-5f86e974-a452-4108-b0b5-e120a4ca2d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:20:19.711894 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 715/1411] 10.222.0.23 () {58 vars in 1015 bytes} [Tue Jun 21 18:20:19 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:20:19.729605 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0353420a-1732-49e5-944d-5b37e0c39fc2 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:19.729896 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0353420a-1732-49e5-944d-5b37e0c39fc2 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:19.730181 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0353420a-1732-49e5-944d-5b37e0c39fc2 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:19.731669 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 697/1412] 10.222.0.23 () {54 vars in 918 bytes} [Tue Jun 21 18:20:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:20:19.738085 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba61c9ed-e199-4f08-80c5-7b68eee8a790 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:19.738419 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba61c9ed-e199-4f08-80c5-7b68eee8a790 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:19.738733 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba61c9ed-e199-4f08-80c5-7b68eee8a790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:19.758448 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-ba61c9ed-e199-4f08-80c5-7b68eee8a790 None None] MFA Rules not processed for user `904255dd3c1f406eb10f6fc65f9cd098`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:20:19.759646 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-ba61c9ed-e199-4f08-80c5-7b68eee8a790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:20:19.769446 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 716/1413] 10.222.0.23 () {58 vars in 1015 bytes} [Tue Jun 21 18:20:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:20:22.389819 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca4b6e-58b5-4f91-802a-f5350e4abbfa None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:22.390529 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca4b6e-58b5-4f91-802a-f5350e4abbfa None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:22.390529 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca4b6e-58b5-4f91-802a-f5350e4abbfa None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:22.392133 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 698/1414] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 18:20:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:20:22.400103 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93552f7-ebfd-4a6e-b5f5-ba772e91d7d6 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:22.400729 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93552f7-ebfd-4a6e-b5f5-ba772e91d7d6 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:22.400729 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93552f7-ebfd-4a6e-b5f5-ba772e91d7d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:22.420667 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-a93552f7-ebfd-4a6e-b5f5-ba772e91d7d6 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:20:22.421936 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-a93552f7-ebfd-4a6e-b5f5-ba772e91d7d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:20:22.430944 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 717/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 18:20:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:20:22.478956 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca4b6e-58b5-4f91-802a-f5350e4abbfa None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:20:22.496230 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca4b6e-58b5-4f91-802a-f5350e4abbfa None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:20:22.503143 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f0c8a5-f376-472c-ad00-2ee31018f16b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904255dd3c1f406eb10f6fc65f9cd098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:20:22.503886 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f0c8a5-f376-472c-ad00-2ee31018f16b service nova] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:22.504218 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f0c8a5-f376-472c-ad00-2ee31018f16b service nova] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:22.504633 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f0c8a5-f376-472c-ad00-2ee31018f16b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:22.514226 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f0c8a5-f376-472c-ad00-2ee31018f16b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:20:22.516160 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f0c8a5-f376-472c-ad00-2ee31018f16b service nova] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:20:22.529250 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 699/1416] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:20:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:20:23.562686 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e4c33-2e11-4ab2-b8dc-58383caea913 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:23.563782 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e4c33-2e11-4ab2-b8dc-58383caea913 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:23.563782 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e4c33-2e11-4ab2-b8dc-58383caea913 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:23.565845 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 718/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 18:20:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:20:23.571125 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dc418d-ed4b-4476-8da6-4904a0a5395e None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:23.571655 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dc418d-ed4b-4476-8da6-4904a0a5395e None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:23.571796 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dc418d-ed4b-4476-8da6-4904a0a5395e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:23.591044 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-19dc418d-ed4b-4476-8da6-4904a0a5395e None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:20:23.592290 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-19dc418d-ed4b-4476-8da6-4904a0a5395e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:20:23.600879 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 700/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 18:20:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:20:23.757980 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6e4c33-2e11-4ab2-b8dc-58383caea913 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:20:23.773765 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6e4c33-2e11-4ab2-b8dc-58383caea913 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:20:23.779460 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f17dfb-8afa-4057-a79a-5e324a2bacb1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904255dd3c1f406eb10f6fc65f9cd098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:20:23.780126 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f17dfb-8afa-4057-a79a-5e324a2bacb1 service nova] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:23.780418 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f17dfb-8afa-4057-a79a-5e324a2bacb1 service nova] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:23.780713 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f17dfb-8afa-4057-a79a-5e324a2bacb1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:23.790432 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f17dfb-8afa-4057-a79a-5e324a2bacb1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:20:23.792028 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f17dfb-8afa-4057-a79a-5e324a2bacb1 service nova] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:20:23.803299 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 719/1419] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:20:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:20:24.795893 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530e16dc-0e54-45cf-9bee-8d8f7236794a None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:24.796590 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530e16dc-0e54-45cf-9bee-8d8f7236794a None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:24.796590 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530e16dc-0e54-45cf-9bee-8d8f7236794a None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:24.799318 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 701/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 18:20:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:20:24.806150 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155fce3b-380f-45ce-a70c-fa39004add64 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:24.806731 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155fce3b-380f-45ce-a70c-fa39004add64 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:24.807127 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155fce3b-380f-45ce-a70c-fa39004add64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:24.832851 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-155fce3b-380f-45ce-a70c-fa39004add64 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:20:24.834187 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-155fce3b-380f-45ce-a70c-fa39004add64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:20:24.842597 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 720/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 18:20:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:20:25.002752 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530e16dc-0e54-45cf-9bee-8d8f7236794a None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:20:25.026360 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530e16dc-0e54-45cf-9bee-8d8f7236794a None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:20:25.035133 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53468a67-7a8d-459f-84b2-6799177207a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904255dd3c1f406eb10f6fc65f9cd098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:20:25.035811 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53468a67-7a8d-459f-84b2-6799177207a3 service nova] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:25.037806 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53468a67-7a8d-459f-84b2-6799177207a3 service nova] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:25.037806 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53468a67-7a8d-459f-84b2-6799177207a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:25.045709 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53468a67-7a8d-459f-84b2-6799177207a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:20:25.047612 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53468a67-7a8d-459f-84b2-6799177207a3 service nova] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:20:25.059242 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 702/1422] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:20:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:20:25.995564 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecfa893-0c3e-4b43-b26c-85dc467a33f7 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:25.996144 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecfa893-0c3e-4b43-b26c-85dc467a33f7 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:25.996144 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecfa893-0c3e-4b43-b26c-85dc467a33f7 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:25.997570 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 721/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 18:20:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:20:26.004378 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26047f6b-99b2-4ca1-9309-731205246d06 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:26.004957 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26047f6b-99b2-4ca1-9309-731205246d06 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:26.005113 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26047f6b-99b2-4ca1-9309-731205246d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:26.024099 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-26047f6b-99b2-4ca1-9309-731205246d06 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:20:26.025399 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-26047f6b-99b2-4ca1-9309-731205246d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:20:26.033843 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 703/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 18:20:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:20:26.189747 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecfa893-0c3e-4b43-b26c-85dc467a33f7 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:20:26.205626 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecfa893-0c3e-4b43-b26c-85dc467a33f7 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:20:26.211570 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc461c-9cc4-48f0-9997-9f1fba522dde service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904255dd3c1f406eb10f6fc65f9cd098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:20:26.212288 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc461c-9cc4-48f0-9997-9f1fba522dde service nova] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:26.212595 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc461c-9cc4-48f0-9997-9f1fba522dde service nova] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:26.212870 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc461c-9cc4-48f0-9997-9f1fba522dde service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:26.223073 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bc461c-9cc4-48f0-9997-9f1fba522dde service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:20:26.225153 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bc461c-9cc4-48f0-9997-9f1fba522dde service nova] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:20:26.237895 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 722/1425] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:20:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:20:27.185949 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e6580-7af4-4338-a397-45b639f48dfe None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:27.186533 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e6580-7af4-4338-a397-45b639f48dfe None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:27.186902 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e6580-7af4-4338-a397-45b639f48dfe None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:27.190145 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 704/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 18:20:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:20:27.195652 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc75de0-8071-4915-8063-b86c89f0c61d None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:27.196160 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc75de0-8071-4915-8063-b86c89f0c61d None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:27.196314 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc75de0-8071-4915-8063-b86c89f0c61d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:27.215817 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-fcc75de0-8071-4915-8063-b86c89f0c61d None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:20:27.217105 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-fcc75de0-8071-4915-8063-b86c89f0c61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:20:27.226970 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 723/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 18:20:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:20:27.384287 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83e6580-7af4-4338-a397-45b639f48dfe None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:20:27.400670 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83e6580-7af4-4338-a397-45b639f48dfe None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:20:27.406642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cd6fff-ef4e-4c57-9161-b0a413b517c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904255dd3c1f406eb10f6fc65f9cd098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:20:27.408787 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd6fff-ef4e-4c57-9161-b0a413b517c3 service nova] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:27.409123 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd6fff-ef4e-4c57-9161-b0a413b517c3 service nova] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:27.409442 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd6fff-ef4e-4c57-9161-b0a413b517c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:27.434966 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cd6fff-ef4e-4c57-9161-b0a413b517c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:20:27.434966 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cd6fff-ef4e-4c57-9161-b0a413b517c3 service nova] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:20:27.436165 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 705/1428] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:20:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:20:28.377222 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e0d2fa-c5e8-4747-8f7b-24684907d0d9 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:28.378037 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e0d2fa-c5e8-4747-8f7b-24684907d0d9 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:28.378037 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e0d2fa-c5e8-4747-8f7b-24684907d0d9 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:28.379211 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 724/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 18:20:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:20:28.385469 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2dd2c5-d86b-4231-82ea-d15815e9c04c None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:28.386080 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2dd2c5-d86b-4231-82ea-d15815e9c04c None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:28.386080 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2dd2c5-d86b-4231-82ea-d15815e9c04c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:28.406257 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-8f2dd2c5-d86b-4231-82ea-d15815e9c04c None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:20:28.408041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-8f2dd2c5-d86b-4231-82ea-d15815e9c04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:20:28.416741 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 706/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 18:20:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:20:28.578417 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e0d2fa-c5e8-4747-8f7b-24684907d0d9 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:20:28.595304 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e0d2fa-c5e8-4747-8f7b-24684907d0d9 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:20:28.601370 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df603cf7-6998-41fe-9689-473c33ce45e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904255dd3c1f406eb10f6fc65f9cd098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:20:28.602041 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df603cf7-6998-41fe-9689-473c33ce45e1 service nova] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:28.602328 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df603cf7-6998-41fe-9689-473c33ce45e1 service nova] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:28.602599 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df603cf7-6998-41fe-9689-473c33ce45e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:28.612704 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df603cf7-6998-41fe-9689-473c33ce45e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:20:28.614609 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df603cf7-6998-41fe-9689-473c33ce45e1 service nova] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:20:28.627359 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 725/1431] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:20:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:20:29.636327 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384db32-44c1-478c-bff1-a8add54f38ac None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:29.636899 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384db32-44c1-478c-bff1-a8add54f38ac None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:29.636899 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384db32-44c1-478c-bff1-a8add54f38ac None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:29.638241 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 707/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 18:20:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:20:29.645147 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805bad6a-dae1-4613-a9c3-6ae53f2cd737 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:29.645719 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805bad6a-dae1-4613-a9c3-6ae53f2cd737 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:29.646110 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805bad6a-dae1-4613-a9c3-6ae53f2cd737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:29.670371 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-805bad6a-dae1-4613-a9c3-6ae53f2cd737 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:20:29.671586 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-805bad6a-dae1-4613-a9c3-6ae53f2cd737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:20:29.680353 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 726/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 18:20:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:20:29.837325 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0384db32-44c1-478c-bff1-a8add54f38ac None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:20:29.864996 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0384db32-44c1-478c-bff1-a8add54f38ac None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:20:29.870950 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64edb587-f687-4962-8934-5b32a3b31529 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904255dd3c1f406eb10f6fc65f9cd098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:20:29.871643 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64edb587-f687-4962-8934-5b32a3b31529 service nova] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:29.871927 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64edb587-f687-4962-8934-5b32a3b31529 service nova] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:29.872204 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64edb587-f687-4962-8934-5b32a3b31529 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:29.881842 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64edb587-f687-4962-8934-5b32a3b31529 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:20:29.883498 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64edb587-f687-4962-8934-5b32a3b31529 service nova] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:20:29.894093 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 708/1434] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:20:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:20:30.795349 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa11b78-7e3a-491f-b1db-d12d70fa0b8b None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:30.796223 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa11b78-7e3a-491f-b1db-d12d70fa0b8b None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:30.796223 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa11b78-7e3a-491f-b1db-d12d70fa0b8b None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:30.797330 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 727/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 18:20:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:20:30.803326 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843052c-76fc-4e76-835b-73cdd8155e58 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:30.803850 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843052c-76fc-4e76-835b-73cdd8155e58 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:30.804009 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843052c-76fc-4e76-835b-73cdd8155e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:30.824551 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-0843052c-76fc-4e76-835b-73cdd8155e58 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:20:30.825783 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-0843052c-76fc-4e76-835b-73cdd8155e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:20:30.834557 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 709/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 18:20:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:20:31.003072 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa11b78-7e3a-491f-b1db-d12d70fa0b8b None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:20:31.018806 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa11b78-7e3a-491f-b1db-d12d70fa0b8b None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:20:31.024925 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9f307f-0c76-4224-8cc9-347d997d63a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904255dd3c1f406eb10f6fc65f9cd098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:20:31.025610 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f307f-0c76-4224-8cc9-347d997d63a0 service nova] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:31.025893 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f307f-0c76-4224-8cc9-347d997d63a0 service nova] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:31.026184 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f307f-0c76-4224-8cc9-347d997d63a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:31.036834 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9f307f-0c76-4224-8cc9-347d997d63a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:20:31.038467 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9f307f-0c76-4224-8cc9-347d997d63a0 service nova] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:20:31.055802 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 728/1437] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:20:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:20:31.953287 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f36a01a-e957-4197-a2a3-ca997b4aef81 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:31.953891 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f36a01a-e957-4197-a2a3-ca997b4aef81 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:31.953891 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f36a01a-e957-4197-a2a3-ca997b4aef81 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:31.955893 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 710/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 18:20:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:20:31.961064 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341e4fcb-ef1a-4cd9-bccc-1a6ae7b642fe None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:31.961515 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341e4fcb-ef1a-4cd9-bccc-1a6ae7b642fe None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:31.961716 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341e4fcb-ef1a-4cd9-bccc-1a6ae7b642fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:31.979374 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-341e4fcb-ef1a-4cd9-bccc-1a6ae7b642fe None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:20:31.980597 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-341e4fcb-ef1a-4cd9-bccc-1a6ae7b642fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:20:31.988497 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 729/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 18:20:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:20:32.147191 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f36a01a-e957-4197-a2a3-ca997b4aef81 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:20:32.162539 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f36a01a-e957-4197-a2a3-ca997b4aef81 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:20:32.168093 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb38226-8b2b-48c9-9141-7e1138f08ad8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904255dd3c1f406eb10f6fc65f9cd098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:20:32.168802 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb38226-8b2b-48c9-9141-7e1138f08ad8 service nova] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:32.169079 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb38226-8b2b-48c9-9141-7e1138f08ad8 service nova] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:32.169352 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb38226-8b2b-48c9-9141-7e1138f08ad8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:32.179678 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb38226-8b2b-48c9-9141-7e1138f08ad8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:20:32.181328 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb38226-8b2b-48c9-9141-7e1138f08ad8 service nova] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:20:32.193101 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 711/1440] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:20:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:20:34.167864 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6bb937-219a-44e3-aede-d62a0e7e4fa9 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:34.168465 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6bb937-219a-44e3-aede-d62a0e7e4fa9 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:34.168465 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6bb937-219a-44e3-aede-d62a0e7e4fa9 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:34.169817 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 730/1441] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 18:20:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:20:34.176158 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b4a3ec-6e12-4e40-935f-2a6ec85a0b87 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:34.176690 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b4a3ec-6e12-4e40-935f-2a6ec85a0b87 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:34.176807 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b4a3ec-6e12-4e40-935f-2a6ec85a0b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:34.194579 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-c3b4a3ec-6e12-4e40-935f-2a6ec85a0b87 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:20:34.195784 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-c3b4a3ec-6e12-4e40-935f-2a6ec85a0b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:20:34.203712 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 712/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 18:20:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:20:34.243327 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6bb937-219a-44e3-aede-d62a0e7e4fa9 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:20:34.254268 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6bb937-219a-44e3-aede-d62a0e7e4fa9 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:20:34.258106 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b7069c-851b-4390-bfd9-3a30a34f60e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:20:34.258755 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b7069c-851b-4390-bfd9-3a30a34f60e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:34.259046 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b7069c-851b-4390-bfd9-3a30a34f60e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:34.259332 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b7069c-851b-4390-bfd9-3a30a34f60e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:34.269137 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b7069c-851b-4390-bfd9-3a30a34f60e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:20:34.270580 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b7069c-851b-4390-bfd9-3a30a34f60e0 service neutron] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:20:34.280287 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 731/1443] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:20:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:20:36.644167 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc7711-6022-44c4-ba02-935b2b3a3048 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:36.644786 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc7711-6022-44c4-ba02-935b2b3a3048 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:36.644786 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc7711-6022-44c4-ba02-935b2b3a3048 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:36.646169 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 713/1444] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 18:20:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:20:36.651733 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95468d3-df7f-4268-a313-199e5cdcabe8 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:36.652201 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95468d3-df7f-4268-a313-199e5cdcabe8 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:36.652301 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95468d3-df7f-4268-a313-199e5cdcabe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:36.670451 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-e95468d3-df7f-4268-a313-199e5cdcabe8 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:20:36.671657 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-e95468d3-df7f-4268-a313-199e5cdcabe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:20:36.679752 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 732/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 18:20:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:20:36.718321 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc7711-6022-44c4-ba02-935b2b3a3048 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:20:36.730176 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc7711-6022-44c4-ba02-935b2b3a3048 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:20:36.733717 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd9db6-9c12-496e-a738-d5e7090143ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:20:36.734363 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd9db6-9c12-496e-a738-d5e7090143ae service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:36.734648 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd9db6-9c12-496e-a738-d5e7090143ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:36.735003 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd9db6-9c12-496e-a738-d5e7090143ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:36.744853 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acd9db6-9c12-496e-a738-d5e7090143ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:20:36.746344 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acd9db6-9c12-496e-a738-d5e7090143ae service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:20:36.756789 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 714/1446] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:20:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:20:39.186293 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e50c2e-814d-41e3-aeef-537cd912c6f2 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:39.186853 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e50c2e-814d-41e3-aeef-537cd912c6f2 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:39.186975 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e50c2e-814d-41e3-aeef-537cd912c6f2 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:39.188336 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 733/1447] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 18:20:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:20:39.194124 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c6b858-e27b-4e09-9223-cb846f26d54e None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:39.199339 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c6b858-e27b-4e09-9223-cb846f26d54e None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:39.199339 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c6b858-e27b-4e09-9223-cb846f26d54e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:39.214215 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-f7c6b858-e27b-4e09-9223-cb846f26d54e None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:20:39.215434 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-f7c6b858-e27b-4e09-9223-cb846f26d54e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:20:39.224530 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 715/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 18:20:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:20:39.265268 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e50c2e-814d-41e3-aeef-537cd912c6f2 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:20:39.277721 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e50c2e-814d-41e3-aeef-537cd912c6f2 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:20:39.281889 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea294c5b-bfe4-4798-bd48-c8d9031bbb49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:20:39.283165 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea294c5b-bfe4-4798-bd48-c8d9031bbb49 service neutron] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:39.283447 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea294c5b-bfe4-4798-bd48-c8d9031bbb49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:39.283752 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea294c5b-bfe4-4798-bd48-c8d9031bbb49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:39.294134 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea294c5b-bfe4-4798-bd48-c8d9031bbb49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:20:39.295591 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea294c5b-bfe4-4798-bd48-c8d9031bbb49 service neutron] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:20:39.308678 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 734/1449] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:20:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:20:41.534552 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e92ee-741e-4600-9974-3070f9ebdba4 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:41.535205 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e92ee-741e-4600-9974-3070f9ebdba4 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:41.535316 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e92ee-741e-4600-9974-3070f9ebdba4 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:41.537008 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 716/1450] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 18:20:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:20:41.542795 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c83b52-2d0c-4505-8a10-239955841067 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:41.543345 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c83b52-2d0c-4505-8a10-239955841067 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:41.543345 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c83b52-2d0c-4505-8a10-239955841067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:41.562321 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-04c83b52-2d0c-4505-8a10-239955841067 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:20:41.564121 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-04c83b52-2d0c-4505-8a10-239955841067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:20:41.572063 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 735/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 18:20:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:20:41.613630 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49e92ee-741e-4600-9974-3070f9ebdba4 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:20:41.624773 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49e92ee-741e-4600-9974-3070f9ebdba4 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:20:41.629149 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e1b0e6-a2e9-4bc9-ab25-01418520a9de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:20:41.629806 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e1b0e6-a2e9-4bc9-ab25-01418520a9de service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:41.630354 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e1b0e6-a2e9-4bc9-ab25-01418520a9de service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:41.630639 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e1b0e6-a2e9-4bc9-ab25-01418520a9de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:41.640717 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e1b0e6-a2e9-4bc9-ab25-01418520a9de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:20:41.643495 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e1b0e6-a2e9-4bc9-ab25-01418520a9de service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:20:41.659433 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 717/1452] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:20:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:20:43.823859 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd1d850-e362-4f8f-b836-59dbef33979a None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:43.824487 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd1d850-e362-4f8f-b836-59dbef33979a None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:43.824487 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd1d850-e362-4f8f-b836-59dbef33979a None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:43.825890 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 736/1453] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 18:20:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:20:43.832172 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1b833-6e60-44b4-bb0a-9f7ad85bb9a1 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:43.832784 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1b833-6e60-44b4-bb0a-9f7ad85bb9a1 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:43.832784 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1b833-6e60-44b4-bb0a-9f7ad85bb9a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:43.851448 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-caa1b833-6e60-44b4-bb0a-9f7ad85bb9a1 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:20:43.853160 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-caa1b833-6e60-44b4-bb0a-9f7ad85bb9a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:20:43.862023 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 718/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 18:20:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:20:43.901432 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd1d850-e362-4f8f-b836-59dbef33979a None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:20:43.912602 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd1d850-e362-4f8f-b836-59dbef33979a None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:20:43.916295 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac7e005-b46a-4691-98a5-a51e589539bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:20:43.917047 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac7e005-b46a-4691-98a5-a51e589539bd service neutron] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:43.917327 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac7e005-b46a-4691-98a5-a51e589539bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:43.917606 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac7e005-b46a-4691-98a5-a51e589539bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:43.928421 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac7e005-b46a-4691-98a5-a51e589539bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:20:43.929901 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac7e005-b46a-4691-98a5-a51e589539bd service neutron] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:20:43.941317 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 737/1455] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:20:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:20:46.081402 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3472c41-ea4a-4fab-8346-24fb74f2a7c3 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:46.081988 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3472c41-ea4a-4fab-8346-24fb74f2a7c3 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:46.081988 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3472c41-ea4a-4fab-8346-24fb74f2a7c3 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:46.083416 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 719/1456] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 18:20:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:20:46.089380 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99018c0d-c668-4d45-9c42-a6e91fcccd7d None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:46.089943 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99018c0d-c668-4d45-9c42-a6e91fcccd7d None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:46.089943 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99018c0d-c668-4d45-9c42-a6e91fcccd7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:46.109360 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-99018c0d-c668-4d45-9c42-a6e91fcccd7d None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:20:46.111875 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-99018c0d-c668-4d45-9c42-a6e91fcccd7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:20:46.120179 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 738/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 18:20:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:20:46.161000 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3472c41-ea4a-4fab-8346-24fb74f2a7c3 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:20:46.173412 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3472c41-ea4a-4fab-8346-24fb74f2a7c3 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:20:46.177158 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e13934-6fb2-4645-b1e6-8fa9ca09d887 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:20:46.177810 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e13934-6fb2-4645-b1e6-8fa9ca09d887 service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:46.178094 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e13934-6fb2-4645-b1e6-8fa9ca09d887 service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:46.178373 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e13934-6fb2-4645-b1e6-8fa9ca09d887 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:46.188587 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e13934-6fb2-4645-b1e6-8fa9ca09d887 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:20:46.190187 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e13934-6fb2-4645-b1e6-8fa9ca09d887 service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:20:46.200917 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 720/1458] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:20:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:20:48.357271 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fda4a2-e54a-47ac-9190-7075459d74b7 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:48.357887 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fda4a2-e54a-47ac-9190-7075459d74b7 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:48.357887 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fda4a2-e54a-47ac-9190-7075459d74b7 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:48.359278 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 739/1459] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 18:20:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:20:48.366301 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b4ca1-7ed0-4132-a833-f7aa5707879a None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:48.366890 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b4ca1-7ed0-4132-a833-f7aa5707879a None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:48.366890 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b4ca1-7ed0-4132-a833-f7aa5707879a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:48.373198 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=70 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:20:48.386152 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-b39b4ca1-7ed0-4132-a833-f7aa5707879a None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:20:48.387407 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-b39b4ca1-7ed0-4132-a833-f7aa5707879a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:20:48.399615 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 721/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 18:20:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:20:48.446544 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fda4a2-e54a-47ac-9190-7075459d74b7 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:20:48.451967 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=73 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:20:48.458300 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fda4a2-e54a-47ac-9190-7075459d74b7 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:20:48.462137 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1069d48-29f9-4ccd-8414-2a8c88de8b52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:20:48.466342 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1069d48-29f9-4ccd-8414-2a8c88de8b52 service neutron] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:48.466342 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1069d48-29f9-4ccd-8414-2a8c88de8b52 service neutron] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:48.466342 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1069d48-29f9-4ccd-8414-2a8c88de8b52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:48.474432 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1069d48-29f9-4ccd-8414-2a8c88de8b52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:20:48.475890 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1069d48-29f9-4ccd-8414-2a8c88de8b52 service neutron] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:20:48.487116 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 740/1461] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:20:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:20:50.629846 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4820a6eb-10d0-4a29-a7a6-a5eaa573ad51 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:50.630394 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4820a6eb-10d0-4a29-a7a6-a5eaa573ad51 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:50.630394 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4820a6eb-10d0-4a29-a7a6-a5eaa573ad51 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:50.631775 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 722/1462] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 18:20:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:20:50.637358 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06075d8-ef59-4775-bb5d-c5e375ca46e1 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:50.637827 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06075d8-ef59-4775-bb5d-c5e375ca46e1 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:50.637946 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06075d8-ef59-4775-bb5d-c5e375ca46e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:50.657620 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-f06075d8-ef59-4775-bb5d-c5e375ca46e1 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:20:50.658749 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-f06075d8-ef59-4775-bb5d-c5e375ca46e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:20:50.666672 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 741/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 18:20:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:20:50.705032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4820a6eb-10d0-4a29-a7a6-a5eaa573ad51 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:20:50.715845 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4820a6eb-10d0-4a29-a7a6-a5eaa573ad51 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:20:50.719988 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946cad7f-608d-4578-b87e-069437c310ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:20:50.720701 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946cad7f-608d-4578-b87e-069437c310ef service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:50.720998 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946cad7f-608d-4578-b87e-069437c310ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:50.722305 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946cad7f-608d-4578-b87e-069437c310ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:50.731896 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946cad7f-608d-4578-b87e-069437c310ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:20:50.733430 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946cad7f-608d-4578-b87e-069437c310ef service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:20:50.744195 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 723/1464] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:20:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:20:52.845183 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04bf042-7869-455f-a65c-442bc27f86d3 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:52.845701 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04bf042-7869-455f-a65c-442bc27f86d3 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:52.845701 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04bf042-7869-455f-a65c-442bc27f86d3 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:52.847159 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 742/1465] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 18:20:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:20:52.852706 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db5883-17b5-4dcd-bc21-05edc497b009 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:52.853163 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db5883-17b5-4dcd-bc21-05edc497b009 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:52.853296 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db5883-17b5-4dcd-bc21-05edc497b009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:52.872479 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-e4db5883-17b5-4dcd-bc21-05edc497b009 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:20:52.873734 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-e4db5883-17b5-4dcd-bc21-05edc497b009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:20:52.884435 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 724/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 18:20:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:20:52.919242 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04bf042-7869-455f-a65c-442bc27f86d3 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:20:52.930850 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04bf042-7869-455f-a65c-442bc27f86d3 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:20:52.934340 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49446d0-a8ec-4321-8555-e635cb7a2984 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:20:52.934981 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49446d0-a8ec-4321-8555-e635cb7a2984 service neutron] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:52.935264 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49446d0-a8ec-4321-8555-e635cb7a2984 service neutron] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:52.935541 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49446d0-a8ec-4321-8555-e635cb7a2984 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:52.945415 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49446d0-a8ec-4321-8555-e635cb7a2984 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:20:52.946843 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49446d0-a8ec-4321-8555-e635cb7a2984 service neutron] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:20:52.957006 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 743/1467] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:20:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:20:55.061382 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff4de2-7996-4455-836d-d81b79f9cfe3 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:55.061953 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff4de2-7996-4455-836d-d81b79f9cfe3 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:55.061953 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff4de2-7996-4455-836d-d81b79f9cfe3 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:55.063938 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 725/1468] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 18:20:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:20:55.069459 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c53e63-b92a-49ed-b754-52b71f59a381 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:55.069905 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c53e63-b92a-49ed-b754-52b71f59a381 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:55.070014 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c53e63-b92a-49ed-b754-52b71f59a381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:55.088114 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-72c53e63-b92a-49ed-b754-52b71f59a381 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:20:55.089339 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-72c53e63-b92a-49ed-b754-52b71f59a381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:20:55.098736 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 744/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 18:20:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:20:55.134908 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ff4de2-7996-4455-836d-d81b79f9cfe3 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:20:55.146309 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ff4de2-7996-4455-836d-d81b79f9cfe3 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:20:55.149787 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fdc716-92a5-4b1d-9068-14b66a51c714 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:20:55.150436 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fdc716-92a5-4b1d-9068-14b66a51c714 service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:20:55.150722 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fdc716-92a5-4b1d-9068-14b66a51c714 service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:20:55.151003 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fdc716-92a5-4b1d-9068-14b66a51c714 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:20:55.160498 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fdc716-92a5-4b1d-9068-14b66a51c714 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:20:55.161937 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fdc716-92a5-4b1d-9068-14b66a51c714 service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:20:55.171180 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 726/1470] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:20:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:21:05.083206 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=17 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:21:05.167140 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=17 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:21:15.899483 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a95b34-45b3-4faf-a829-fec810e8d64c None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:15.900113 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a95b34-45b3-4faf-a829-fec810e8d64c None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:15.900331 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a95b34-45b3-4faf-a829-fec810e8d64c None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:15.902786 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 745/1471] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:21:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:21:15.909726 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40627ad0-69cb-4bad-be07-33a0f8b43130 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:15.910392 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40627ad0-69cb-4bad-be07-33a0f8b43130 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:15.910392 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40627ad0-69cb-4bad-be07-33a0f8b43130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:15.929639 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-40627ad0-69cb-4bad-be07-33a0f8b43130 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:21:15.930883 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-40627ad0-69cb-4bad-be07-33a0f8b43130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:21:15.939563 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 727/1472] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:21:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:21:15.989491 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aeca6f-2650-4af4-bedc-fbd9223eedbb None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:15.989783 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aeca6f-2650-4af4-bedc-fbd9223eedbb None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:15.990068 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aeca6f-2650-4af4-bedc-fbd9223eedbb None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:15.992029 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 746/1473] 10.222.0.23 () {54 vars in 962 bytes} [Tue Jun 21 18:21:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:21:15.997346 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f2a68-75e2-456c-8867-977113f40af6 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:15.997711 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f2a68-75e2-456c-8867-977113f40af6 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:15.998035 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f2a68-75e2-456c-8867-977113f40af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:16.018504 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-407f2a68-75e2-456c-8867-977113f40af6 None None] MFA Rules not processed for user `9672794bc1ca4dcd8259d51d9290384b`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:21:16.019743 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-407f2a68-75e2-456c-8867-977113f40af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:21:16.026611 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 728/1474] 10.222.0.23 () {58 vars in 1059 bytes} [Tue Jun 21 18:21:15 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:21:16.035438 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aeca6f-2650-4af4-bedc-fbd9223eedbb None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:21:16.047797 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aeca6f-2650-4af4-bedc-fbd9223eedbb None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:21:16.051641 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20733f9-2c53-40ab-97fe-4052722b14a6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9672794bc1ca4dcd8259d51d9290384b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:21:16.052302 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20733f9-2c53-40ab-97fe-4052722b14a6 service glance] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:16.052620 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20733f9-2c53-40ab-97fe-4052722b14a6 service glance] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:16.052907 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20733f9-2c53-40ab-97fe-4052722b14a6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:16.063430 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20733f9-2c53-40ab-97fe-4052722b14a6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:21:16.064925 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20733f9-2c53-40ab-97fe-4052722b14a6 service glance] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:21:16.075334 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 747/1475] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:21:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:21:18.814186 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b05bd-49ca-4c82-8d7b-954b46932cf7 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:18.814835 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b05bd-49ca-4c82-8d7b-954b46932cf7 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:18.814835 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b05bd-49ca-4c82-8d7b-954b46932cf7 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:18.816123 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 729/1476] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:21:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:21:18.824150 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6bdb0b-21f5-4b15-9d99-9b0397ebf7ef None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:18.824732 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6bdb0b-21f5-4b15-9d99-9b0397ebf7ef None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:18.824978 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6bdb0b-21f5-4b15-9d99-9b0397ebf7ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:18.846131 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-ae6bdb0b-21f5-4b15-9d99-9b0397ebf7ef None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:21:18.847364 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-ae6bdb0b-21f5-4b15-9d99-9b0397ebf7ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:21:18.855853 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 748/1477] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:21:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:21:18.882453 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69b05bd-49ca-4c82-8d7b-954b46932cf7 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:21:18.900842 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69b05bd-49ca-4c82-8d7b-954b46932cf7 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:21:18.908650 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6267d9-51c7-4f3e-b0a6-05f2a4073ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:21:18.909777 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6267d9-51c7-4f3e-b0a6-05f2a4073ff5 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:18.910274 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6267d9-51c7-4f3e-b0a6-05f2a4073ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:18.910752 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6267d9-51c7-4f3e-b0a6-05f2a4073ff5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:18.912003 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6267d9-51c7-4f3e-b0a6-05f2a4073ff5 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:21:18.915205 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: WARNING py.warnings [None req-7c6267d9-51c7-4f3e-b0a6-05f2a4073ff5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 21 18:21:18.915205 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: warnings.warn(msg) Jun 21 18:21:18.915205 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: Jun 21 18:21:18.917601 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6267d9-51c7-4f3e-b0a6-05f2a4073ff5 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:21:18.957834 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 730/1478] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jun 21 18:21:18 2022] GET /identity/v3/projects => generated 2553 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:21:21.124492 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e66c68-0ab7-4501-b6c0-e5ca790fcf92 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:21.125068 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e66c68-0ab7-4501-b6c0-e5ca790fcf92 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:21.125068 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e66c68-0ab7-4501-b6c0-e5ca790fcf92 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:21.126425 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 749/1479] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:21:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:21:21.132002 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb36036-7940-4c65-8379-d21eb455dc6c None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:21.132544 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb36036-7940-4c65-8379-d21eb455dc6c None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:21.132663 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb36036-7940-4c65-8379-d21eb455dc6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:21.151751 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-dbb36036-7940-4c65-8379-d21eb455dc6c None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:21:21.153058 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-dbb36036-7940-4c65-8379-d21eb455dc6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:21:21.162427 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 731/1480] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:21:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:21:21.207905 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e66c68-0ab7-4501-b6c0-e5ca790fcf92 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:21:21.222199 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e66c68-0ab7-4501-b6c0-e5ca790fcf92 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:21:21.227739 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d03db5-eb87-47d9-97a4-006d893e4552 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904255dd3c1f406eb10f6fc65f9cd098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:21:21.228396 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d03db5-eb87-47d9-97a4-006d893e4552 service nova] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:21.228694 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d03db5-eb87-47d9-97a4-006d893e4552 service nova] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:21.228970 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d03db5-eb87-47d9-97a4-006d893e4552 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:21.238232 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d03db5-eb87-47d9-97a4-006d893e4552 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:21:21.239823 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d03db5-eb87-47d9-97a4-006d893e4552 service nova] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:21:21.251064 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 750/1481] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:21:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:21:23.555114 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a352b06-f939-447d-8acf-c0466a9dd0c5 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:23.558599 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a352b06-f939-447d-8acf-c0466a9dd0c5 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:23.558599 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a352b06-f939-447d-8acf-c0466a9dd0c5 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:23.558599 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 732/1482] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:21:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:21:23.564220 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7634bba1-e5e1-4e05-bb9f-27e1823407a1 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:23.564886 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7634bba1-e5e1-4e05-bb9f-27e1823407a1 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:23.564886 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7634bba1-e5e1-4e05-bb9f-27e1823407a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:23.583180 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-7634bba1-e5e1-4e05-bb9f-27e1823407a1 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:21:23.584454 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-7634bba1-e5e1-4e05-bb9f-27e1823407a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:21:23.592254 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 751/1483] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:21:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:21:23.633336 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a352b06-f939-447d-8acf-c0466a9dd0c5 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:21:23.647515 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a352b06-f939-447d-8acf-c0466a9dd0c5 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:21:23.651310 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09425df-9873-47b0-965f-acd779aff54e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:21:23.651976 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09425df-9873-47b0-965f-acd779aff54e service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:23.652281 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09425df-9873-47b0-965f-acd779aff54e service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:23.652642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09425df-9873-47b0-965f-acd779aff54e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:23.662730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09425df-9873-47b0-965f-acd779aff54e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:21:23.664178 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09425df-9873-47b0-965f-acd779aff54e service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:21:23.675250 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 733/1484] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:21:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:21:25.819955 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de455d5c-8725-44ac-96d1-86682f42f4ba None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:25.820586 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de455d5c-8725-44ac-96d1-86682f42f4ba None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:25.820793 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de455d5c-8725-44ac-96d1-86682f42f4ba None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:25.822234 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 752/1485] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:21:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:21:25.828701 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8421d9-4582-4a7d-ba3b-63fbc44d5841 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:25.829457 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8421d9-4582-4a7d-ba3b-63fbc44d5841 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:25.829618 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8421d9-4582-4a7d-ba3b-63fbc44d5841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:25.848446 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-2b8421d9-4582-4a7d-ba3b-63fbc44d5841 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:21:25.849688 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-2b8421d9-4582-4a7d-ba3b-63fbc44d5841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:21:25.857843 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 734/1486] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:21:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:21:25.900446 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de455d5c-8725-44ac-96d1-86682f42f4ba None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:21:25.902055 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-de455d5c-8725-44ac-96d1-86682f42f4ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:21:25.915977 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de455d5c-8725-44ac-96d1-86682f42f4ba None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:21:25.919579 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bb89f5-5abd-44ad-bbe6-1f9f1db8cb75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:21:25.920237 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb89f5-5abd-44ad-bbe6-1f9f1db8cb75 service neutron] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:25.920556 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb89f5-5abd-44ad-bbe6-1f9f1db8cb75 service neutron] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:25.920829 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb89f5-5abd-44ad-bbe6-1f9f1db8cb75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:25.932043 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bb89f5-5abd-44ad-bbe6-1f9f1db8cb75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:21:25.933509 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bb89f5-5abd-44ad-bbe6-1f9f1db8cb75 service neutron] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:21:25.944818 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 753/1487] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:21:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:21:28.218534 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1fd2b-bdfa-48a7-b5e0-fc544626304d None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:28.219195 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1fd2b-bdfa-48a7-b5e0-fc544626304d None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:28.219785 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1fd2b-bdfa-48a7-b5e0-fc544626304d None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:28.221203 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 735/1488] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:21:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:21:28.227227 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc86f9-03d7-41a7-a194-7cc2217346fd None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:28.227775 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc86f9-03d7-41a7-a194-7cc2217346fd None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:28.227775 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc86f9-03d7-41a7-a194-7cc2217346fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:28.246024 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-7afc86f9-03d7-41a7-a194-7cc2217346fd None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:21:28.247214 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-7afc86f9-03d7-41a7-a194-7cc2217346fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:21:28.255986 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 754/1489] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:21:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:21:28.297281 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a1fd2b-bdfa-48a7-b5e0-fc544626304d None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:21:28.308831 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a1fd2b-bdfa-48a7-b5e0-fc544626304d None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:21:28.312497 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d53ae5-8fe4-460c-8ce2-90b7edf8d05d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:21:28.313975 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d53ae5-8fe4-460c-8ce2-90b7edf8d05d service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:28.314261 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d53ae5-8fe4-460c-8ce2-90b7edf8d05d service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:28.314539 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d53ae5-8fe4-460c-8ce2-90b7edf8d05d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:28.323994 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d53ae5-8fe4-460c-8ce2-90b7edf8d05d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:21:28.325470 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d53ae5-8fe4-460c-8ce2-90b7edf8d05d service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:21:28.336428 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 736/1490] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:21:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:21:30.944224 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047aa882-a325-40de-8f24-65158e919a52 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:30.944923 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047aa882-a325-40de-8f24-65158e919a52 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:30.944923 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047aa882-a325-40de-8f24-65158e919a52 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:30.946265 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 755/1491] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jun 21 18:21:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:21:30.954198 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4552450c-21db-4ba2-8151-3da9c18a49bb None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:30.959460 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4552450c-21db-4ba2-8151-3da9c18a49bb None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:30.959460 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4552450c-21db-4ba2-8151-3da9c18a49bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:30.982251 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-4552450c-21db-4ba2-8151-3da9c18a49bb None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:21:30.984795 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-4552450c-21db-4ba2-8151-3da9c18a49bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:21:30.994753 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 737/1492] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jun 21 18:21:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:21:31.040954 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047aa882-a325-40de-8f24-65158e919a52 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:21:31.057236 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047aa882-a325-40de-8f24-65158e919a52 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:21:31.066945 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768d211f-03e8-46ea-b6e3-28df4d020eeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:21:31.066945 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d211f-03e8-46ea-b6e3-28df4d020eeb service neutron] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:21:31.066945 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d211f-03e8-46ea-b6e3-28df4d020eeb service neutron] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:21:31.066945 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d211f-03e8-46ea-b6e3-28df4d020eeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:21:31.075772 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768d211f-03e8-46ea-b6e3-28df4d020eeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:21:31.077174 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768d211f-03e8-46ea-b6e3-28df4d020eeb service neutron] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:21:31.088152 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 756/1493] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:21:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:21:40.970321 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=33 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:21:41.083108 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=33 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:23:13.365047 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf1e64d-b309-484e-9914-7965cf4c1771 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:23:13.365682 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf1e64d-b309-484e-9914-7965cf4c1771 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:23:13.365682 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf1e64d-b309-484e-9914-7965cf4c1771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:23:13.385628 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-2bf1e64d-b309-484e-9914-7965cf4c1771 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:23:13.386831 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-2bf1e64d-b309-484e-9914-7965cf4c1771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:23:13.395330 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 738/1494] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:23:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:23:13.404631 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca13f06-6614-41e2-985d-391847658cfb None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:23:13.405169 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca13f06-6614-41e2-985d-391847658cfb None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:23:13.405774 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca13f06-6614-41e2-985d-391847658cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:23:13.424151 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-7ca13f06-6614-41e2-985d-391847658cfb None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:23:13.425367 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-7ca13f06-6614-41e2-985d-391847658cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:23:13.433147 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 757/1495] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:23:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:23:13.438813 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf1e64d-b309-484e-9914-7965cf4c1771 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:23:13.448496 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf1e64d-b309-484e-9914-7965cf4c1771 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:23:13.452989 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed400ca-3102-44ba-99e7-a545e5e916c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:23:13.453645 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed400ca-3102-44ba-99e7-a545e5e916c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:23:13.453930 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed400ca-3102-44ba-99e7-a545e5e916c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:23:13.454207 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed400ca-3102-44ba-99e7-a545e5e916c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:23:13.455673 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed400ca-3102-44ba-99e7-a545e5e916c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:23:13.455999 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed400ca-3102-44ba-99e7-a545e5e916c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:23:13.457117 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: WARNING py.warnings [None req-6ed400ca-3102-44ba-99e7-a545e5e916c7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 21 18:23:13.457117 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: warnings.warn(msg) Jun 21 18:23:13.457117 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: Jun 21 18:23:13.458018 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed400ca-3102-44ba-99e7-a545e5e916c7 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:23:13.469785 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 739/1496] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:23:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:23:13.477912 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca13f06-6614-41e2-985d-391847658cfb None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:23:13.487911 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca13f06-6614-41e2-985d-391847658cfb None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:23:13.492538 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd6a083-ece3-4498-9d35-27ac8aea0b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:23:13.493192 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd6a083-ece3-4498-9d35-27ac8aea0b03 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:23:13.493477 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd6a083-ece3-4498-9d35-27ac8aea0b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:23:13.494037 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd6a083-ece3-4498-9d35-27ac8aea0b03 admin admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:23:13.494729 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd6a083-ece3-4498-9d35-27ac8aea0b03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:23:13.496503 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: WARNING py.warnings [None req-8dd6a083-ece3-4498-9d35-27ac8aea0b03 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 21 18:23:13.496503 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: warnings.warn(msg) Jun 21 18:23:13.496503 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: Jun 21 18:23:13.497689 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd6a083-ece3-4498-9d35-27ac8aea0b03 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:23:13.609935 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 758/1497] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:23:13 2022] POST /identity/v3/projects => generated 379 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:23:13.614346 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed400ca-3102-44ba-99e7-a545e5e916c7 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:23:13.625889 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed400ca-3102-44ba-99e7-a545e5e916c7 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:23:13.680851 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f90f04-5dcb-4efd-a871-197f284c5e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:23:13.681529 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f90f04-5dcb-4efd-a871-197f284c5e38 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:23:13.681817 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f90f04-5dcb-4efd-a871-197f284c5e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:23:13.682094 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f90f04-5dcb-4efd-a871-197f284c5e38 admin admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:23:13.682790 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f90f04-5dcb-4efd-a871-197f284c5e38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:23:13.683832 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: WARNING py.warnings [None req-33f90f04-5dcb-4efd-a871-197f284c5e38 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 21 18:23:13.683832 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: warnings.warn(msg) Jun 21 18:23:13.683832 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: Jun 21 18:23:13.685051 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f90f04-5dcb-4efd-a871-197f284c5e38 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:23:14.167840 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 740/1498] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:23:13 2022] POST /identity/v3/users => generated 290 bytes in 554 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:23:14.172130 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd6a083-ece3-4498-9d35-27ac8aea0b03 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:23:14.182342 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd6a083-ece3-4498-9d35-27ac8aea0b03 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:23:14.187206 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f440bb9f-61cb-43d5-809c-8b308a78c622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:23:14.187874 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440bb9f-61cb-43d5-809c-8b308a78c622 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:23:14.188160 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440bb9f-61cb-43d5-809c-8b308a78c622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:23:14.188475 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440bb9f-61cb-43d5-809c-8b308a78c622 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:23:14.189200 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f440bb9f-61cb-43d5-809c-8b308a78c622 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:23:14.190220 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: WARNING py.warnings [None req-f440bb9f-61cb-43d5-809c-8b308a78c622 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 21 18:23:14.190220 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: warnings.warn(msg) Jun 21 18:23:14.190220 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: Jun 21 18:23:14.191098 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f440bb9f-61cb-43d5-809c-8b308a78c622 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:23:14.204553 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 759/1499] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:23:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:23:14.209195 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f90f04-5dcb-4efd-a871-197f284c5e38 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:23:14.220334 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f90f04-5dcb-4efd-a871-197f284c5e38 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:23:14.224949 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0907683c-68e6-4e28-b940-0e79f65575ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:23:14.225634 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0907683c-68e6-4e28-b940-0e79f65575ae admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:23:14.225921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0907683c-68e6-4e28-b940-0e79f65575ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:23:14.226204 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0907683c-68e6-4e28-b940-0e79f65575ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:23:14.226944 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0907683c-68e6-4e28-b940-0e79f65575ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:23:14.227969 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: WARNING py.warnings [None req-0907683c-68e6-4e28-b940-0e79f65575ae admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 21 18:23:14.227969 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: warnings.warn(msg) Jun 21 18:23:14.227969 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: Jun 21 18:23:14.228956 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0907683c-68e6-4e28-b940-0e79f65575ae admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:23:14.243266 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 741/1500] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:23:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:23:14.247805 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f440bb9f-61cb-43d5-809c-8b308a78c622 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:23:14.258434 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f440bb9f-61cb-43d5-809c-8b308a78c622 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:23:14.263143 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298f6beb-527a-4ff2-a336-d23885ec2084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:23:14.264663 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f6beb-527a-4ff2-a336-d23885ec2084 admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:23:14.265066 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f6beb-527a-4ff2-a336-d23885ec2084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:23:14.265625 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f6beb-527a-4ff2-a336-d23885ec2084 admin admin] PATH_INFO: `/v3/projects/e63c6170f0624f8180450cbd18162ffd/users/56f2baf778844cc8a5eae6b76d55a8c0/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:23:14.281877 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298f6beb-527a-4ff2-a336-d23885ec2084 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e63c6170f0624f8180450cbd18162ffd, user_id=56f2baf778844cc8a5eae6b76d55a8c0, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:23:14.282920 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: WARNING py.warnings [None req-298f6beb-527a-4ff2-a336-d23885ec2084 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 21 18:23:14.282920 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: warnings.warn(msg) Jun 21 18:23:14.282920 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: Jun 21 18:23:14.284857 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298f6beb-527a-4ff2-a336-d23885ec2084 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:23:14.394641 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 760/1501] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:23:14 2022] PUT /identity/v3/projects/e63c6170f0624f8180450cbd18162ffd/users/56f2baf778844cc8a5eae6b76d55a8c0/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:23:14.406874 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71fdf4c-6038-4ae6-b384-df657e32d8b7 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:23:14.410014 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71fdf4c-6038-4ae6-b384-df657e32d8b7 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:23:14.410014 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71fdf4c-6038-4ae6-b384-df657e32d8b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:23:14.438454 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-e71fdf4c-6038-4ae6-b384-df657e32d8b7 None None] MFA Rules not processed for user `56f2baf778844cc8a5eae6b76d55a8c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:23:14.439688 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-e71fdf4c-6038-4ae6-b384-df657e32d8b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:23:14.511885 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 742/1502] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:23:14 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:23:14.525755 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7434ac71-978b-404f-b7ef-b5548a95aee6 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:23:14.526279 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7434ac71-978b-404f-b7ef-b5548a95aee6 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:23:14.526372 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7434ac71-978b-404f-b7ef-b5548a95aee6 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:23:14.527730 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 761/1503] 10.222.0.23 () {54 vars in 963 bytes} [Tue Jun 21 18:23:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:23:14.533630 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f11540-3ce8-4505-9551-8b1078edae40 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:23:14.533921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f11540-3ce8-4505-9551-8b1078edae40 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:23:14.534198 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f11540-3ce8-4505-9551-8b1078edae40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:23:14.554841 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-20f11540-3ce8-4505-9551-8b1078edae40 None None] MFA Rules not processed for user `77c6e43555944902a22ba2fe17a8e99f`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:23:14.556080 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-20f11540-3ce8-4505-9551-8b1078edae40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:23:14.603547 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:23:14.611965 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 743/1504] 10.222.0.23 () {58 vars in 1060 bytes} [Tue Jun 21 18:23:14 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:23:14.619968 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7434ac71-978b-404f-b7ef-b5548a95aee6 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:23:14.630955 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7434ac71-978b-404f-b7ef-b5548a95aee6 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:23:14.634562 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9f197c-7449-49f1-9642-fd8534cfa5a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:23:14.635210 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9f197c-7449-49f1-9642-fd8534cfa5a2 service cinder] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:23:14.635492 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9f197c-7449-49f1-9642-fd8534cfa5a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:23:14.635770 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9f197c-7449-49f1-9642-fd8534cfa5a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:23:14.644776 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9f197c-7449-49f1-9642-fd8534cfa5a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:23:14.646217 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9f197c-7449-49f1-9642-fd8534cfa5a2 service cinder] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:23:14.656149 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 762/1505] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:23:14 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:23:14.699031 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f5e378-dd5e-465d-b232-fac7c5063fc6 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:23:14.699317 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f5e378-dd5e-465d-b232-fac7c5063fc6 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:23:14.699591 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f5e378-dd5e-465d-b232-fac7c5063fc6 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:23:14.700935 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 744/1506] 10.222.0.23 () {52 vars in 829 bytes} [Tue Jun 21 18:23:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:23:14.704613 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9f197c-7449-49f1-9642-fd8534cfa5a2 service cinder] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:23:14.769437 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9f197c-7449-49f1-9642-fd8534cfa5a2 service cinder] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:23:14.774276 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfac8b8-3207-4400-a386-4dfe6cb63c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:23:14.775157 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfac8b8-3207-4400-a386-4dfe6cb63c5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:23:14.775430 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfac8b8-3207-4400-a386-4dfe6cb63c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:23:14.775706 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfac8b8-3207-4400-a386-4dfe6cb63c5f admin admin] PATH_INFO: `/v3/users/56f2baf778844cc8a5eae6b76d55a8c0` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:23:14.777317 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfac8b8-3207-4400-a386-4dfe6cb63c5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=56f2baf778844cc8a5eae6b76d55a8c0)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:23:14.778315 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: WARNING py.warnings [None req-3dfac8b8-3207-4400-a386-4dfe6cb63c5f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 21 18:23:14.778315 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: warnings.warn(msg) Jun 21 18:23:14.778315 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: Jun 21 18:23:14.779533 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfac8b8-3207-4400-a386-4dfe6cb63c5f admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:23:15.005841 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-3dfac8b8-3207-4400-a386-4dfe6cb63c5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56f2baf778844cc8a5eae6b76d55a8c0'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:23:15.025517 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-3dfac8b8-3207-4400-a386-4dfe6cb63c5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56f2baf778844cc8a5eae6b76d55a8c0'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:23:15.087814 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-3dfac8b8-3207-4400-a386-4dfe6cb63c5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56f2baf778844cc8a5eae6b76d55a8c0'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:23:15.088474 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-3dfac8b8-3207-4400-a386-4dfe6cb63c5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56f2baf778844cc8a5eae6b76d55a8c0'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:23:15.099225 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:23:15.103329 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-3dfac8b8-3207-4400-a386-4dfe6cb63c5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56f2baf778844cc8a5eae6b76d55a8c0'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:23:15.109804 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 763/1507] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:23:14 2022] DELETE /identity/v3/users/56f2baf778844cc8a5eae6b76d55a8c0 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:23:15.114507 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf9a71b-df6c-4710-a222-48e8a1a2ac5b None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:23:15.114800 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf9a71b-df6c-4710-a222-48e8a1a2ac5b None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:23:15.115080 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf9a71b-df6c-4710-a222-48e8a1a2ac5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:23:15.133275 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-bdf9a71b-df6c-4710-a222-48e8a1a2ac5b None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:23:15.134496 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-bdf9a71b-df6c-4710-a222-48e8a1a2ac5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:23:15.193960 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 745/1508] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:23:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:23:15.204284 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfac8b8-3207-4400-a386-4dfe6cb63c5f admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:23:15.205700 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-3dfac8b8-3207-4400-a386-4dfe6cb63c5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:23:15.265202 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfac8b8-3207-4400-a386-4dfe6cb63c5f admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:23:15.268880 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35516ff1-e259-4c01-98d2-6caf3d51fcc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:23:15.269535 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35516ff1-e259-4c01-98d2-6caf3d51fcc7 service neutron] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:23:15.269968 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35516ff1-e259-4c01-98d2-6caf3d51fcc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:23:15.270204 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35516ff1-e259-4c01-98d2-6caf3d51fcc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:23:15.279330 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35516ff1-e259-4c01-98d2-6caf3d51fcc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:23:15.281777 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35516ff1-e259-4c01-98d2-6caf3d51fcc7 service neutron] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:23:15.291253 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 764/1509] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:23:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:23:15.790764 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf9a71b-df6c-4710-a222-48e8a1a2ac5b None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:23:15.792336 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-bdf9a71b-df6c-4710-a222-48e8a1a2ac5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:23:15.805025 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf9a71b-df6c-4710-a222-48e8a1a2ac5b None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:23:15.810276 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e63ddc-d7e1-47a2-be7b-f86cd2a28900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:23:15.811795 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e63ddc-d7e1-47a2-be7b-f86cd2a28900 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:23:15.812079 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e63ddc-d7e1-47a2-be7b-f86cd2a28900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:23:15.812358 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e63ddc-d7e1-47a2-be7b-f86cd2a28900 admin admin] PATH_INFO: `/v3/projects/e63c6170f0624f8180450cbd18162ffd` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:23:15.813627 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e63ddc-d7e1-47a2-be7b-f86cd2a28900 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e63c6170f0624f8180450cbd18162ffd)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:23:15.814661 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: WARNING py.warnings [None req-f6e63ddc-d7e1-47a2-be7b-f86cd2a28900 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 21 18:23:15.814661 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: warnings.warn(msg) Jun 21 18:23:15.814661 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: Jun 21 18:23:15.815866 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e63ddc-d7e1-47a2-be7b-f86cd2a28900 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:23:15.946017 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-f6e63ddc-d7e1-47a2-be7b-f86cd2a28900 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e63c6170f0624f8180450cbd18162ffd', 'request_id': 'req-f6e63ddc-d7e1-47a2-be7b-f86cd2a28900'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:23:16.007405 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-f6e63ddc-d7e1-47a2-be7b-f86cd2a28900 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e63c6170f0624f8180450cbd18162ffd', 'request_id': 'req-f6e63ddc-d7e1-47a2-be7b-f86cd2a28900'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:23:16.014256 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-f6e63ddc-d7e1-47a2-be7b-f86cd2a28900 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e63c6170f0624f8180450cbd18162ffd', 'request_id': 'req-f6e63ddc-d7e1-47a2-be7b-f86cd2a28900'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:23:16.034067 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-f6e63ddc-d7e1-47a2-be7b-f86cd2a28900 admin admin] The token cache is being invalidate because project e63c6170f0624f8180450cbd18162ffd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:23:16.034309 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-f6e63ddc-d7e1-47a2-be7b-f86cd2a28900 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:23:16.035085 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-f6e63ddc-d7e1-47a2-be7b-f86cd2a28900 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:23:16.036767 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 746/1510] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:23:15 2022] DELETE /identity/v3/projects/e63c6170f0624f8180450cbd18162ffd => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:23:25.286973 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=29 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:23:26.030696 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:24:57.423148 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350a2543-8cc6-433c-95f7-54c03e7290da None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:24:57.423784 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350a2543-8cc6-433c-95f7-54c03e7290da None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:24:57.423784 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350a2543-8cc6-433c-95f7-54c03e7290da None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:24:57.425159 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 765/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 18:24:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:24:57.430204 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ebba65-7f79-46a2-b2c0-9f5f5359f630 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:24:57.430641 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ebba65-7f79-46a2-b2c0-9f5f5359f630 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:24:57.430835 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ebba65-7f79-46a2-b2c0-9f5f5359f630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:24:57.450299 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-d1ebba65-7f79-46a2-b2c0-9f5f5359f630 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:24:57.451616 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-d1ebba65-7f79-46a2-b2c0-9f5f5359f630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:24:57.633687 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 747/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 18:24:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:24:57.790309 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350a2543-8cc6-433c-95f7-54c03e7290da None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:24:57.792053 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-350a2543-8cc6-433c-95f7-54c03e7290da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:24:57.866697 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350a2543-8cc6-433c-95f7-54c03e7290da None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:24:57.872334 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f8a706-f68c-4d5c-b1f0-47ffa650b850 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904255dd3c1f406eb10f6fc65f9cd098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:24:57.873009 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f8a706-f68c-4d5c-b1f0-47ffa650b850 service nova] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:24:57.873297 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f8a706-f68c-4d5c-b1f0-47ffa650b850 service nova] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:24:57.873598 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f8a706-f68c-4d5c-b1f0-47ffa650b850 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:24:57.883723 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f8a706-f68c-4d5c-b1f0-47ffa650b850 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:24:57.885392 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f8a706-f68c-4d5c-b1f0-47ffa650b850 service nova] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:24:57.896206 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 766/1513] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:24:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:24:58.741602 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d939bb-18c2-4cf6-b807-fdc3c9ee1869 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:24:58.742194 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d939bb-18c2-4cf6-b807-fdc3c9ee1869 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:24:58.742194 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d939bb-18c2-4cf6-b807-fdc3c9ee1869 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:24:58.743545 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 748/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 18:24:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:24:58.749102 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41d647-bcb8-45a4-b575-cc3502e619f6 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:24:58.750202 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41d647-bcb8-45a4-b575-cc3502e619f6 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:24:58.750401 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41d647-bcb8-45a4-b575-cc3502e619f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:24:58.768227 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-2c41d647-bcb8-45a4-b575-cc3502e619f6 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:24:58.769463 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-2c41d647-bcb8-45a4-b575-cc3502e619f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:24:58.777628 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 767/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 18:24:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:24:58.931202 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d939bb-18c2-4cf6-b807-fdc3c9ee1869 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:24:58.947758 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d939bb-18c2-4cf6-b807-fdc3c9ee1869 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:24:58.953939 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e480a-debf-4f82-8b23-08d549995871 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904255dd3c1f406eb10f6fc65f9cd098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:24:58.954612 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e480a-debf-4f82-8b23-08d549995871 service nova] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:24:58.954898 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e480a-debf-4f82-8b23-08d549995871 service nova] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:24:58.955196 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e480a-debf-4f82-8b23-08d549995871 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:24:58.964306 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83e480a-debf-4f82-8b23-08d549995871 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:24:58.965998 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83e480a-debf-4f82-8b23-08d549995871 service nova] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:24:58.976372 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 749/1516] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:24:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:24:59.372002 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aa9cef-8c19-42a7-8731-4ffe645b06c4 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:24:59.372294 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aa9cef-8c19-42a7-8731-4ffe645b06c4 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:24:59.372515 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aa9cef-8c19-42a7-8731-4ffe645b06c4 None None] PATH_INFO: `/v3` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:24:59.373872 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 768/1517] 10.222.0.72 () {54 vars in 925 bytes} [Tue Jun 21 18:24:59 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:24:59.379930 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db66f0-0522-449a-ad12-ae883bb0d798 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:24:59.380337 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db66f0-0522-449a-ad12-ae883bb0d798 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:24:59.380532 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db66f0-0522-449a-ad12-ae883bb0d798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:24:59.400133 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-24db66f0-0522-449a-ad12-ae883bb0d798 None None] MFA Rules not processed for user `938187967efe49be9c5389d1feef81ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:24:59.401414 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-24db66f0-0522-449a-ad12-ae883bb0d798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:24:59.465310 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 750/1518] 10.222.0.72 () {58 vars in 1015 bytes} [Tue Jun 21 18:24:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:25:03.435583 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d0bcc2-d3aa-476e-bb95-dc299f9d155a None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:25:03.436248 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d0bcc2-d3aa-476e-bb95-dc299f9d155a None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:25:03.436371 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d0bcc2-d3aa-476e-bb95-dc299f9d155a None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:25:03.438031 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 769/1519] 10.222.0.23 () {54 vars in 960 bytes} [Tue Jun 21 18:25:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:25:03.447734 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c45da1-c96e-4157-a682-563d042c5e96 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:25:03.448279 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c45da1-c96e-4157-a682-563d042c5e96 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:25:03.448279 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c45da1-c96e-4157-a682-563d042c5e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:25:03.467028 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-40c45da1-c96e-4157-a682-563d042c5e96 None None] MFA Rules not processed for user `938187967efe49be9c5389d1feef81ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:25:03.468231 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-40c45da1-c96e-4157-a682-563d042c5e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:25:03.477421 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 751/1520] 10.222.0.23 () {58 vars in 1057 bytes} [Tue Jun 21 18:25:03 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:25:03.485239 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d0bcc2-d3aa-476e-bb95-dc299f9d155a None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:25:03.498747 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d0bcc2-d3aa-476e-bb95-dc299f9d155a None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:25:03.504087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab97c17-07f1-4908-a68f-18679759a561 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938187967efe49be9c5389d1feef81ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:25:03.504849 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab97c17-07f1-4908-a68f-18679759a561 service placement] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:25:03.505135 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab97c17-07f1-4908-a68f-18679759a561 service placement] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:25:03.505412 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab97c17-07f1-4908-a68f-18679759a561 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:25:03.514238 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab97c17-07f1-4908-a68f-18679759a561 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:25:03.516444 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab97c17-07f1-4908-a68f-18679759a561 service placement] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:25:03.526360 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 770/1521] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:25:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:25:10.225569 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42c71ff-1c9a-4b8e-958b-21af57fca0b6 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:25:10.226429 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42c71ff-1c9a-4b8e-958b-21af57fca0b6 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:25:10.226594 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42c71ff-1c9a-4b8e-958b-21af57fca0b6 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:25:10.228050 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 752/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 18:25:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:25:10.233102 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcac714-a238-4c71-935e-b0fb45b5ba43 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:25:10.233542 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcac714-a238-4c71-935e-b0fb45b5ba43 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:25:10.234187 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcac714-a238-4c71-935e-b0fb45b5ba43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:25:10.252893 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-9fcac714-a238-4c71-935e-b0fb45b5ba43 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:25:10.254085 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-9fcac714-a238-4c71-935e-b0fb45b5ba43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:25:10.262026 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 771/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 18:25:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:25:10.417482 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42c71ff-1c9a-4b8e-958b-21af57fca0b6 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:25:10.432563 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42c71ff-1c9a-4b8e-958b-21af57fca0b6 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:25:10.438416 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20be1d5-36c1-46f2-8a23-2ffd3eb4d75a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904255dd3c1f406eb10f6fc65f9cd098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:25:10.439064 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20be1d5-36c1-46f2-8a23-2ffd3eb4d75a service nova] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:25:10.439341 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20be1d5-36c1-46f2-8a23-2ffd3eb4d75a service nova] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:25:10.439613 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20be1d5-36c1-46f2-8a23-2ffd3eb4d75a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:25:10.448281 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20be1d5-36c1-46f2-8a23-2ffd3eb4d75a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:25:10.449884 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20be1d5-36c1-46f2-8a23-2ffd3eb4d75a service nova] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:25:10.461724 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 753/1524] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:25:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:25:11.301528 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88280d90-8976-4cd8-87c6-beed4d900d0f None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:25:11.302105 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88280d90-8976-4cd8-87c6-beed4d900d0f None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:25:11.302105 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88280d90-8976-4cd8-87c6-beed4d900d0f None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:25:11.303474 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 772/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 18:25:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:25:11.308500 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d6093-6aee-4fa5-bc7f-7fc4ca6be24a None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:25:11.309070 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d6093-6aee-4fa5-bc7f-7fc4ca6be24a None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:25:11.309070 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d6093-6aee-4fa5-bc7f-7fc4ca6be24a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:25:11.326755 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-562d6093-6aee-4fa5-bc7f-7fc4ca6be24a None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:25:11.328021 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-562d6093-6aee-4fa5-bc7f-7fc4ca6be24a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:25:11.336008 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 754/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 18:25:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:25:11.488608 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88280d90-8976-4cd8-87c6-beed4d900d0f None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:25:11.503059 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88280d90-8976-4cd8-87c6-beed4d900d0f None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:25:11.508731 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f95a5d8-4cd6-47a4-9695-77746a0b1729 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904255dd3c1f406eb10f6fc65f9cd098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:25:11.509417 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f95a5d8-4cd6-47a4-9695-77746a0b1729 service nova] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:25:11.509700 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f95a5d8-4cd6-47a4-9695-77746a0b1729 service nova] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:25:11.509987 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f95a5d8-4cd6-47a4-9695-77746a0b1729 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:25:11.519200 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f95a5d8-4cd6-47a4-9695-77746a0b1729 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:25:11.520824 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f95a5d8-4cd6-47a4-9695-77746a0b1729 service nova] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:25:11.530490 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 773/1527] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:25:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:25:15.841332 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce069b2d-f94a-43ae-b685-8bf69827ffa4 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:25:15.841965 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce069b2d-f94a-43ae-b685-8bf69827ffa4 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:25:15.841965 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce069b2d-f94a-43ae-b685-8bf69827ffa4 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:25:15.843916 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 755/1528] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 18:25:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:25:15.849654 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db6372-2c8e-4d63-b2d4-95d2a4447b4f None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:25:15.850135 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db6372-2c8e-4d63-b2d4-95d2a4447b4f None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:25:15.850244 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db6372-2c8e-4d63-b2d4-95d2a4447b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:25:15.868977 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-f6db6372-2c8e-4d63-b2d4-95d2a4447b4f None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:25:15.870169 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-f6db6372-2c8e-4d63-b2d4-95d2a4447b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:25:15.878636 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 774/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 18:25:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:25:15.916607 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce069b2d-f94a-43ae-b685-8bf69827ffa4 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:25:15.918308 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-ce069b2d-f94a-43ae-b685-8bf69827ffa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:25:15.976193 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce069b2d-f94a-43ae-b685-8bf69827ffa4 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:25:15.979748 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab26423f-5ed5-4644-b497-8bf1da1767f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:25:15.980417 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab26423f-5ed5-4644-b497-8bf1da1767f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:25:15.980711 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab26423f-5ed5-4644-b497-8bf1da1767f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:25:15.980986 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab26423f-5ed5-4644-b497-8bf1da1767f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:25:15.990713 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab26423f-5ed5-4644-b497-8bf1da1767f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:25:15.992134 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab26423f-5ed5-4644-b497-8bf1da1767f6 service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:25:15.997344 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:25:16.002485 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 756/1530] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:25:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:25:18.050852 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d26e3-9ed4-47bc-8c39-6d7a9f83026c None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:25:18.051488 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d26e3-9ed4-47bc-8c39-6d7a9f83026c None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:25:18.051488 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d26e3-9ed4-47bc-8c39-6d7a9f83026c None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:25:18.052807 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 775/1531] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 18:25:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:25:18.058107 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aefc303-6cf8-4de6-a7fd-813e65b17f01 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:25:18.058534 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aefc303-6cf8-4de6-a7fd-813e65b17f01 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:25:18.058671 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aefc303-6cf8-4de6-a7fd-813e65b17f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:25:18.076667 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-9aefc303-6cf8-4de6-a7fd-813e65b17f01 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:25:18.077925 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-9aefc303-6cf8-4de6-a7fd-813e65b17f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:25:18.085777 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 757/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 18:25:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:25:18.122932 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274d26e3-9ed4-47bc-8c39-6d7a9f83026c None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:25:18.133718 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274d26e3-9ed4-47bc-8c39-6d7a9f83026c None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:25:18.137293 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7828b3c4-3745-4b41-b061-cee38800a314 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:25:18.137917 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828b3c4-3745-4b41-b061-cee38800a314 service neutron] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:25:18.138175 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828b3c4-3745-4b41-b061-cee38800a314 service neutron] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:25:18.138452 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828b3c4-3745-4b41-b061-cee38800a314 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:25:18.147573 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7828b3c4-3745-4b41-b061-cee38800a314 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:25:18.149053 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7828b3c4-3745-4b41-b061-cee38800a314 service neutron] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:25:18.159400 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 776/1533] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:25:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:25:21.891444 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b57f12-3963-4755-8096-58d56972a63a None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:25:21.891880 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b57f12-3963-4755-8096-58d56972a63a None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:25:21.892035 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b57f12-3963-4755-8096-58d56972a63a None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:25:21.893474 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 758/1534] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 18:25:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:25:21.898961 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f097e162-c700-46fb-b947-59db00874c53 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:25:21.899415 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f097e162-c700-46fb-b947-59db00874c53 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:25:21.899518 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f097e162-c700-46fb-b947-59db00874c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:25:21.918039 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-f097e162-c700-46fb-b947-59db00874c53 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:25:21.919246 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-f097e162-c700-46fb-b947-59db00874c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:25:21.928011 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 777/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 18:25:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:25:22.247182 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b57f12-3963-4755-8096-58d56972a63a None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:25:22.248928 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-27b57f12-3963-4755-8096-58d56972a63a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:25:22.306338 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b57f12-3963-4755-8096-58d56972a63a None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:25:22.309831 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e4db7-9713-422b-84dd-dcf6605ba73f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:25:22.310478 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e4db7-9713-422b-84dd-dcf6605ba73f service cinder] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:25:22.310762 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e4db7-9713-422b-84dd-dcf6605ba73f service cinder] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:25:22.311038 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e4db7-9713-422b-84dd-dcf6605ba73f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:25:22.320482 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9e4db7-9713-422b-84dd-dcf6605ba73f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:25:22.321900 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9e4db7-9713-422b-84dd-dcf6605ba73f service cinder] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:25:22.331197 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 759/1536] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:25:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:25:24.447053 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c68e2-b352-4a0a-b22f-0b507e84ecd7 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:25:24.447633 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c68e2-b352-4a0a-b22f-0b507e84ecd7 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:25:24.447633 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c68e2-b352-4a0a-b22f-0b507e84ecd7 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:25:24.449022 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 778/1537] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 18:25:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:25:24.455085 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17709c1f-d201-4c92-bbe1-00f0c482bc52 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:25:24.455600 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17709c1f-d201-4c92-bbe1-00f0c482bc52 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:25:24.455716 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17709c1f-d201-4c92-bbe1-00f0c482bc52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:25:24.474141 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-17709c1f-d201-4c92-bbe1-00f0c482bc52 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:25:24.475503 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-17709c1f-d201-4c92-bbe1-00f0c482bc52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:25:24.483639 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 760/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 18:25:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:25:24.794086 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4c68e2-b352-4a0a-b22f-0b507e84ecd7 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:25:24.797223 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-3d4c68e2-b352-4a0a-b22f-0b507e84ecd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:25:24.810800 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4c68e2-b352-4a0a-b22f-0b507e84ecd7 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:25:24.814420 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127e848a-4e7c-4482-b119-a0b466642753 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:25:24.815072 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127e848a-4e7c-4482-b119-a0b466642753 service cinder] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:25:24.815351 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127e848a-4e7c-4482-b119-a0b466642753 service cinder] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:25:24.815627 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127e848a-4e7c-4482-b119-a0b466642753 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:25:24.825046 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127e848a-4e7c-4482-b119-a0b466642753 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:25:24.826494 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127e848a-4e7c-4482-b119-a0b466642753 service cinder] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:25:24.836469 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 779/1539] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:25:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:25:34.469433 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=32 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:25:34.832086 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=70 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:26:54.840976 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9cd545-1bb6-4199-a347-7207f64d5f7e None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:54.842386 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9cd545-1bb6-4199-a347-7207f64d5f7e None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:54.842793 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9cd545-1bb6-4199-a347-7207f64d5f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:54.876843 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-da9cd545-1bb6-4199-a347-7207f64d5f7e None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:26:54.878398 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-da9cd545-1bb6-4199-a347-7207f64d5f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:26:54.901363 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 761/1540] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:26:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:26:54.918460 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a678da-9f27-4594-84f0-398e30bcaefa None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:54.918460 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a678da-9f27-4594-84f0-398e30bcaefa None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:54.918460 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a678da-9f27-4594-84f0-398e30bcaefa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:54.949476 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-f1a678da-9f27-4594-84f0-398e30bcaefa None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:26:54.950980 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-f1a678da-9f27-4594-84f0-398e30bcaefa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:26:54.964926 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 780/1541] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:26:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:26:54.969650 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9cd545-1bb6-4199-a347-7207f64d5f7e None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:54.980929 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9cd545-1bb6-4199-a347-7207f64d5f7e None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:54.993611 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7055efd8-d57a-4d45-bb2d-73705e000fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:54.994982 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055efd8-d57a-4d45-bb2d-73705e000fe7 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:54.995431 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055efd8-d57a-4d45-bb2d-73705e000fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:54.995888 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055efd8-d57a-4d45-bb2d-73705e000fe7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:54.997328 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7055efd8-d57a-4d45-bb2d-73705e000fe7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:26:54.997888 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7055efd8-d57a-4d45-bb2d-73705e000fe7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:55.001482 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7055efd8-d57a-4d45-bb2d-73705e000fe7 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:55.016715 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0677cb7c-af4e-4c37-bd19-43a5eae61c45 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:55.017061 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0677cb7c-af4e-4c37-bd19-43a5eae61c45 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:55.017292 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0677cb7c-af4e-4c37-bd19-43a5eae61c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:55.037060 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 762/1542] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:26:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:26:55.049425 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7055efd8-d57a-4d45-bb2d-73705e000fe7 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:55.055333 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-0677cb7c-af4e-4c37-bd19-43a5eae61c45 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:26:55.056660 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-0677cb7c-af4e-4c37-bd19-43a5eae61c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:26:55.076320 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 781/1543] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:26:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:26:55.076684 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7055efd8-d57a-4d45-bb2d-73705e000fe7 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:55.087544 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51393bf-9226-4f87-b892-fb46cd64e163 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:55.087899 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51393bf-9226-4f87-b892-fb46cd64e163 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:55.088114 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51393bf-9226-4f87-b892-fb46cd64e163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:55.092220 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454d5c0e-9d8f-4b24-bb17-7e2aab75a2e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:55.093437 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d5c0e-9d8f-4b24-bb17-7e2aab75a2e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:55.093888 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d5c0e-9d8f-4b24-bb17-7e2aab75a2e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:55.094330 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d5c0e-9d8f-4b24-bb17-7e2aab75a2e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:55.096279 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454d5c0e-9d8f-4b24-bb17-7e2aab75a2e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:55.098132 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: WARNING py.warnings [None req-454d5c0e-9d8f-4b24-bb17-7e2aab75a2e4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 21 18:26:55.098132 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: warnings.warn(msg) Jun 21 18:26:55.098132 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: Jun 21 18:26:55.100210 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454d5c0e-9d8f-4b24-bb17-7e2aab75a2e4 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:55.127403 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-c51393bf-9226-4f87-b892-fb46cd64e163 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:26:55.129585 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-c51393bf-9226-4f87-b892-fb46cd64e163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:26:55.141966 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 782/1544] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:26:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:26:55.147080 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51393bf-9226-4f87-b892-fb46cd64e163 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:55.160635 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51393bf-9226-4f87-b892-fb46cd64e163 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:55.167710 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d235dce-29ea-49bc-b36f-ba95689ded5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:55.168896 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d235dce-29ea-49bc-b36f-ba95689ded5a admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:55.169470 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d235dce-29ea-49bc-b36f-ba95689ded5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:55.170056 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d235dce-29ea-49bc-b36f-ba95689ded5a admin admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:55.171285 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d235dce-29ea-49bc-b36f-ba95689ded5a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:26:55.171836 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d235dce-29ea-49bc-b36f-ba95689ded5a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:55.173275 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: WARNING py.warnings [None req-9d235dce-29ea-49bc-b36f-ba95689ded5a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 21 18:26:55.173275 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: warnings.warn(msg) Jun 21 18:26:55.173275 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: Jun 21 18:26:55.174643 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d235dce-29ea-49bc-b36f-ba95689ded5a admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:55.194152 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 783/1545] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:26:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:26:55.196724 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e3fb3-0fa0-4c84-b226-077560fc6fc8 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:55.197284 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e3fb3-0fa0-4c84-b226-077560fc6fc8 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:55.197804 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e3fb3-0fa0-4c84-b226-077560fc6fc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:55.217222 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 763/1546] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:26:55 2022] POST /identity/v3/projects => generated 385 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:26:55.218480 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454d5c0e-9d8f-4b24-bb17-7e2aab75a2e4 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:55.248998 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-533e3fb3-0fa0-4c84-b226-077560fc6fc8 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:26:55.250660 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-533e3fb3-0fa0-4c84-b226-077560fc6fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:26:55.385975 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454d5c0e-9d8f-4b24-bb17-7e2aab75a2e4 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:55.406137 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915eeda0-2135-4bc7-95af-b19f3c4032d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:55.406545 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 784/1547] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:26:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:26:55.407199 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915eeda0-2135-4bc7-95af-b19f3c4032d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:55.407709 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915eeda0-2135-4bc7-95af-b19f3c4032d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:55.408210 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915eeda0-2135-4bc7-95af-b19f3c4032d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:55.408522 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533e3fb3-0fa0-4c84-b226-077560fc6fc8 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:55.409583 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915eeda0-2135-4bc7-95af-b19f3c4032d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:55.421597 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915eeda0-2135-4bc7-95af-b19f3c4032d0 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:55.424622 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533e3fb3-0fa0-4c84-b226-077560fc6fc8 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:55.437845 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f0ad99-69b9-4631-9bf0-7cd80e323352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:55.439143 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f0ad99-69b9-4631-9bf0-7cd80e323352 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:55.439671 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f0ad99-69b9-4631-9bf0-7cd80e323352 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:55.440172 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f0ad99-69b9-4631-9bf0-7cd80e323352 admin admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:55.441145 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f0ad99-69b9-4631-9bf0-7cd80e323352 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:55.442405 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: WARNING py.warnings [None req-d7f0ad99-69b9-4631-9bf0-7cd80e323352 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 21 18:26:55.442405 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: warnings.warn(msg) Jun 21 18:26:55.442405 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: Jun 21 18:26:55.444064 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f0ad99-69b9-4631-9bf0-7cd80e323352 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:55.569268 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 785/1548] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:26:55 2022] POST /identity/v3/users => generated 293 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:26:55.570849 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed085339-e45b-4bc3-8d6c-4503e0cd653d None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:55.571852 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed085339-e45b-4bc3-8d6c-4503e0cd653d None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:55.572231 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed085339-e45b-4bc3-8d6c-4503e0cd653d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:55.592692 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-ed085339-e45b-4bc3-8d6c-4503e0cd653d None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:26:55.593895 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-ed085339-e45b-4bc3-8d6c-4503e0cd653d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:26:55.604431 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 786/1549] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:26:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:26:55.606377 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137694fb-29dd-4692-ae80-28022cdfdd95 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:55.606669 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137694fb-29dd-4692-ae80-28022cdfdd95 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:55.606948 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137694fb-29dd-4692-ae80-28022cdfdd95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:55.630297 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-137694fb-29dd-4692-ae80-28022cdfdd95 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:26:55.632518 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-137694fb-29dd-4692-ae80-28022cdfdd95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:26:55.641675 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 787/1550] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:26:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:26:55.642882 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db9a2d0-b19b-4d0c-9b3c-23428f4e27de None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:55.644772 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db9a2d0-b19b-4d0c-9b3c-23428f4e27de None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:55.645234 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db9a2d0-b19b-4d0c-9b3c-23428f4e27de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:55.658244 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 764/1551] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:26:55 2022] POST /identity/v3/projects => generated 365 bytes in 440 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:26:55.659610 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fee8f45-6467-4922-9af7-f8b6a5156876 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:55.659993 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fee8f45-6467-4922-9af7-f8b6a5156876 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:55.660403 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fee8f45-6467-4922-9af7-f8b6a5156876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:55.675093 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-3db9a2d0-b19b-4d0c-9b3c-23428f4e27de None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:26:55.676758 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-3db9a2d0-b19b-4d0c-9b3c-23428f4e27de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:26:55.681250 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-5fee8f45-6467-4922-9af7-f8b6a5156876 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:26:55.692329 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-5fee8f45-6467-4922-9af7-f8b6a5156876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:26:55.763286 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 788/1552] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:26:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:26:55.768953 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 765/1553] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:26:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:26:55.768953 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fee8f45-6467-4922-9af7-f8b6a5156876 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:55.771189 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaba473-08cd-4729-9cb5-42d7abc13f79 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:55.771651 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaba473-08cd-4729-9cb5-42d7abc13f79 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:55.772094 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaba473-08cd-4729-9cb5-42d7abc13f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:55.778995 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fee8f45-6467-4922-9af7-f8b6a5156876 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:55.784198 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ade40b-2682-4c36-9b30-f05ac798b623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:55.785008 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ade40b-2682-4c36-9b30-f05ac798b623 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:55.785395 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ade40b-2682-4c36-9b30-f05ac798b623 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:55.785772 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ade40b-2682-4c36-9b30-f05ac798b623 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:55.786606 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ade40b-2682-4c36-9b30-f05ac798b623 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:55.788318 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ade40b-2682-4c36-9b30-f05ac798b623 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:55.796431 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-2aaba473-08cd-4729-9cb5-42d7abc13f79 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:26:55.797659 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-2aaba473-08cd-4729-9cb5-42d7abc13f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:26:55.799397 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 766/1554] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:26:55 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:26:55.803456 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ade40b-2682-4c36-9b30-f05ac798b623 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:55.811220 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 789/1555] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:26:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:26:55.811220 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaba473-08cd-4729-9cb5-42d7abc13f79 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:55.817084 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ade40b-2682-4c36-9b30-f05ac798b623 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:55.822241 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaba473-08cd-4729-9cb5-42d7abc13f79 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:55.824023 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f3a03f-b68c-4c6f-8859-341b27229e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:55.824023 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f3a03f-b68c-4c6f-8859-341b27229e8f admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:55.824267 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f3a03f-b68c-4c6f-8859-341b27229e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:55.824654 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f3a03f-b68c-4c6f-8859-341b27229e8f admin admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:55.825496 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f3a03f-b68c-4c6f-8859-341b27229e8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:26:55.825900 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f3a03f-b68c-4c6f-8859-341b27229e8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:55.827374 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21d04d7-c084-4c96-a9b0-096a4ee08784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:55.827752 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f3a03f-b68c-4c6f-8859-341b27229e8f admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:55.828084 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21d04d7-c084-4c96-a9b0-096a4ee08784 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:55.828369 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21d04d7-c084-4c96-a9b0-096a4ee08784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:55.828676 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21d04d7-c084-4c96-a9b0-096a4ee08784 admin admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:55.829383 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21d04d7-c084-4c96-a9b0-096a4ee08784 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:55.833477 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21d04d7-c084-4c96-a9b0-096a4ee08784 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:55.845721 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 767/1556] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:26:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:26:55.847345 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc009a83-cd0f-4fbf-a2ef-1fd21dbd7ddd None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:55.847491 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc009a83-cd0f-4fbf-a2ef-1fd21dbd7ddd None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:55.847775 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc009a83-cd0f-4fbf-a2ef-1fd21dbd7ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:55.866235 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-dc009a83-cd0f-4fbf-a2ef-1fd21dbd7ddd None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:26:55.867427 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-dc009a83-cd0f-4fbf-a2ef-1fd21dbd7ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:26:55.875895 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 768/1557] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:26:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:26:55.877144 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e822540c-6512-4f33-b37b-2bfe657f4fcd None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:55.877429 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e822540c-6512-4f33-b37b-2bfe657f4fcd None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:55.877704 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e822540c-6512-4f33-b37b-2bfe657f4fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:55.896428 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-e822540c-6512-4f33-b37b-2bfe657f4fcd None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:26:55.897600 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-e822540c-6512-4f33-b37b-2bfe657f4fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:26:55.906218 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 769/1558] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:26:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:26:55.915869 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae1363-6745-4ca5-be7b-958b2640bcf2 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:55.915869 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae1363-6745-4ca5-be7b-958b2640bcf2 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:55.915869 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae1363-6745-4ca5-be7b-958b2640bcf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:55.949685 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:26:55.955768 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-d2ae1363-6745-4ca5-be7b-958b2640bcf2 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:26:55.957003 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-d2ae1363-6745-4ca5-be7b-958b2640bcf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:26:55.965791 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 770/1559] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:26:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:26:55.966635 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ae1363-6745-4ca5-be7b-958b2640bcf2 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:55.975914 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ae1363-6745-4ca5-be7b-958b2640bcf2 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:55.980940 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8d083a-5e4c-475e-adf1-399365278fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:55.981595 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8d083a-5e4c-475e-adf1-399365278fdd admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:55.981882 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8d083a-5e4c-475e-adf1-399365278fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:55.982158 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8d083a-5e4c-475e-adf1-399365278fdd admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:55.982884 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8d083a-5e4c-475e-adf1-399365278fdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:55.984501 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8d083a-5e4c-475e-adf1-399365278fdd admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:55.994443 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 771/1560] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:26:55 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:26:55.995269 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8d083a-5e4c-475e-adf1-399365278fdd admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:56.004943 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8d083a-5e4c-475e-adf1-399365278fdd admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:56.010910 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfba255-c281-4aa2-b726-c6a48f5829a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:56.011545 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfba255-c281-4aa2-b726-c6a48f5829a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:56.011835 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfba255-c281-4aa2-b726-c6a48f5829a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:56.012259 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfba255-c281-4aa2-b726-c6a48f5829a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:56.013015 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfba255-c281-4aa2-b726-c6a48f5829a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:26:56.013326 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfba255-c281-4aa2-b726-c6a48f5829a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:56.014890 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfba255-c281-4aa2-b726-c6a48f5829a6 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:56.028318 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 772/1561] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:26:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:26:56.029621 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9951e269-cda0-48c7-8690-079c30f4b65a None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:56.029912 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9951e269-cda0-48c7-8690-079c30f4b65a None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:56.030193 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9951e269-cda0-48c7-8690-079c30f4b65a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:56.048328 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-9951e269-cda0-48c7-8690-079c30f4b65a None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:26:56.049578 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-9951e269-cda0-48c7-8690-079c30f4b65a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:26:56.058866 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 773/1562] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:26:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:26:56.059671 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9951e269-cda0-48c7-8690-079c30f4b65a None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:56.069845 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9951e269-cda0-48c7-8690-079c30f4b65a None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:56.075359 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238951e5-2a29-4fb2-a5c7-da259e47581a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:56.076006 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238951e5-2a29-4fb2-a5c7-da259e47581a admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:56.076334 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238951e5-2a29-4fb2-a5c7-da259e47581a admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:56.076601 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238951e5-2a29-4fb2-a5c7-da259e47581a admin admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:56.077297 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238951e5-2a29-4fb2-a5c7-da259e47581a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:56.079188 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238951e5-2a29-4fb2-a5c7-da259e47581a admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:56.107190 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 790/1563] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:26:55 2022] POST /identity/v3/users => generated 283 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:26:56.108540 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f23ac31-8e6c-4439-9f05-e68c8271f692 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:56.108840 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f23ac31-8e6c-4439-9f05-e68c8271f692 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:56.109118 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f23ac31-8e6c-4439-9f05-e68c8271f692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:56.128232 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-1f23ac31-8e6c-4439-9f05-e68c8271f692 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:26:56.129482 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-1f23ac31-8e6c-4439-9f05-e68c8271f692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:26:56.138491 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 791/1564] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:26:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:26:56.139270 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f23ac31-8e6c-4439-9f05-e68c8271f692 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:56.149115 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f23ac31-8e6c-4439-9f05-e68c8271f692 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:56.153504 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2ea667-7028-4538-803f-0c99363c32e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:56.154181 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 774/1565] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:26:56 2022] POST /identity/v3/projects => generated 405 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:26:56.154313 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ea667-7028-4538-803f-0c99363c32e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:56.154497 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ea667-7028-4538-803f-0c99363c32e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:56.154708 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ea667-7028-4538-803f-0c99363c32e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:56.155204 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238951e5-2a29-4fb2-a5c7-da259e47581a admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:56.155545 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2ea667-7028-4538-803f-0c99363c32e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:26:56.155880 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2ea667-7028-4538-803f-0c99363c32e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:56.157791 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2ea667-7028-4538-803f-0c99363c32e5 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:56.163109 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:26:56.170809 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 792/1566] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:26:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:26:56.171614 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2ea667-7028-4538-803f-0c99363c32e5 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:56.182206 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2ea667-7028-4538-803f-0c99363c32e5 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:56.219889 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238951e5-2a29-4fb2-a5c7-da259e47581a admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:56.224526 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64815066-5681-4ce4-a4d3-dd84b1dd20f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:56.225199 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64815066-5681-4ce4-a4d3-dd84b1dd20f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:56.225481 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64815066-5681-4ce4-a4d3-dd84b1dd20f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:56.225757 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64815066-5681-4ce4-a4d3-dd84b1dd20f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:56.226478 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64815066-5681-4ce4-a4d3-dd84b1dd20f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:26:56.226788 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64815066-5681-4ce4-a4d3-dd84b1dd20f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:56.228441 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64815066-5681-4ce4-a4d3-dd84b1dd20f9 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:56.239795 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3cb0f3-ed72-4cb3-88df-8ca6de700a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:56.240292 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 775/1567] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:26:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:26:56.240856 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3cb0f3-ed72-4cb3-88df-8ca6de700a70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:56.240986 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64815066-5681-4ce4-a4d3-dd84b1dd20f9 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:56.241150 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3cb0f3-ed72-4cb3-88df-8ca6de700a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:56.241432 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3cb0f3-ed72-4cb3-88df-8ca6de700a70 admin admin] PATH_INFO: `/v3/projects/d3bfd39123f64e27805ea90036c1340c/users/61b74b7ef05740fdad199df473316eea/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:56.251290 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64815066-5681-4ce4-a4d3-dd84b1dd20f9 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:56.255925 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a73a8f9-24b7-415b-8860-9b965683dcbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:56.256666 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a73a8f9-24b7-415b-8860-9b965683dcbe admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:56.256961 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a73a8f9-24b7-415b-8860-9b965683dcbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:56.257240 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a73a8f9-24b7-415b-8860-9b965683dcbe admin admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:56.257795 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3cb0f3-ed72-4cb3-88df-8ca6de700a70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3bfd39123f64e27805ea90036c1340c, user_id=61b74b7ef05740fdad199df473316eea, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:56.257912 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a73a8f9-24b7-415b-8860-9b965683dcbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:56.260599 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a73a8f9-24b7-415b-8860-9b965683dcbe admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:56.261251 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3cb0f3-ed72-4cb3-88df-8ca6de700a70 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:56.316749 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 793/1568] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:26:56 2022] PUT /identity/v3/projects/d3bfd39123f64e27805ea90036c1340c/users/61b74b7ef05740fdad199df473316eea/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:26:56.318221 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e9aafe-0628-436b-97fc-173439d68683 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:56.318516 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e9aafe-0628-436b-97fc-173439d68683 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:56.318794 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e9aafe-0628-436b-97fc-173439d68683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:56.336691 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-45e9aafe-0628-436b-97fc-173439d68683 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:26:56.337904 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-45e9aafe-0628-436b-97fc-173439d68683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:26:56.341809 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 776/1569] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:26:56 2022] POST /identity/v3/projects => generated 377 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:26:56.342709 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a73a8f9-24b7-415b-8860-9b965683dcbe admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:56.397974 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 794/1570] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:26:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:26:56.398875 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e9aafe-0628-436b-97fc-173439d68683 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:56.408306 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e9aafe-0628-436b-97fc-173439d68683 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:56.412227 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a73a8f9-24b7-415b-8860-9b965683dcbe admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:56.412951 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f232048d-42d6-48c5-bc72-865718813e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:56.413603 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f232048d-42d6-48c5-bc72-865718813e0b admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:56.413885 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f232048d-42d6-48c5-bc72-865718813e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:56.414178 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f232048d-42d6-48c5-bc72-865718813e0b admin admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:56.415459 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f232048d-42d6-48c5-bc72-865718813e0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:26:56.415769 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f232048d-42d6-48c5-bc72-865718813e0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:56.417399 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f232048d-42d6-48c5-bc72-865718813e0b admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:56.417543 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2be941-18bc-4b39-8967-11032aba4f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:56.418191 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2be941-18bc-4b39-8967-11032aba4f23 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:56.418475 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2be941-18bc-4b39-8967-11032aba4f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:56.418877 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2be941-18bc-4b39-8967-11032aba4f23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:56.419479 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2be941-18bc-4b39-8967-11032aba4f23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:56.421107 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2be941-18bc-4b39-8967-11032aba4f23 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:56.430847 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 795/1571] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:26:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:26:56.431659 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f232048d-42d6-48c5-bc72-865718813e0b admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:56.434850 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 777/1572] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:26:56.434850 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2be941-18bc-4b39-8967-11032aba4f23 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:56.444780 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2be941-18bc-4b39-8967-11032aba4f23 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:56.444946 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f232048d-42d6-48c5-bc72-865718813e0b admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:56.449618 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a69be3d-497a-4442-8dd7-68b932a76d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:56.449780 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb63869-4452-4783-bf6c-e11db1e5b256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:56.450288 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a69be3d-497a-4442-8dd7-68b932a76d5b admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:56.450397 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb63869-4452-4783-bf6c-e11db1e5b256 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:56.450577 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a69be3d-497a-4442-8dd7-68b932a76d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:56.450700 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb63869-4452-4783-bf6c-e11db1e5b256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:56.450857 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a69be3d-497a-4442-8dd7-68b932a76d5b admin admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:56.450956 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb63869-4452-4783-bf6c-e11db1e5b256 admin admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:56.451559 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a69be3d-497a-4442-8dd7-68b932a76d5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:56.451681 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb63869-4452-4783-bf6c-e11db1e5b256 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:56.453500 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a69be3d-497a-4442-8dd7-68b932a76d5b admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:56.453617 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb63869-4452-4783-bf6c-e11db1e5b256 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:56.469968 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:26:56.547023 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 796/1573] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:26:56 2022] POST /identity/v3/users => generated 303 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:26:56.547847 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a69be3d-497a-4442-8dd7-68b932a76d5b admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:56.558833 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a69be3d-497a-4442-8dd7-68b932a76d5b admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:56.564003 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615676ef-8153-4452-a142-ebde74bd7bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:56.566811 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615676ef-8153-4452-a142-ebde74bd7bbd admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:56.566811 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615676ef-8153-4452-a142-ebde74bd7bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:56.566811 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615676ef-8153-4452-a142-ebde74bd7bbd admin admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:56.566811 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615676ef-8153-4452-a142-ebde74bd7bbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:56.567824 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615676ef-8153-4452-a142-ebde74bd7bbd admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:56.575074 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 778/1574] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:26:56 2022] POST /identity/v3/projects => generated 393 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:26:56.575074 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea69a3d-de49-4170-bc1a-12c25b64bdbe None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:56.575074 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea69a3d-de49-4170-bc1a-12c25b64bdbe None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:56.575074 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea69a3d-de49-4170-bc1a-12c25b64bdbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:56.600088 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-fea69a3d-de49-4170-bc1a-12c25b64bdbe None None] MFA Rules not processed for user `61b74b7ef05740fdad199df473316eea`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:26:56.601372 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-fea69a3d-de49-4170-bc1a-12c25b64bdbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:26:56.666196 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 779/1575] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:26:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:26:56.667084 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea69a3d-de49-4170-bc1a-12c25b64bdbe None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:56.679554 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 797/1576] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:26:56 2022] POST /identity/v3/projects => generated 403 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:26:56.687179 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615676ef-8153-4452-a142-ebde74bd7bbd admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:56.712970 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:26:56.735653 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea69a3d-de49-4170-bc1a-12c25b64bdbe None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:56.750104 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615676ef-8153-4452-a142-ebde74bd7bbd admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:56.754743 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11061591-3861-4e82-bd0a-d6ed61310f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:56.755399 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11061591-3861-4e82-bd0a-d6ed61310f8f admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:56.755682 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11061591-3861-4e82-bd0a-d6ed61310f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:56.755957 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11061591-3861-4e82-bd0a-d6ed61310f8f admin admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:56.756752 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11061591-3861-4e82-bd0a-d6ed61310f8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:26:56.757056 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11061591-3861-4e82-bd0a-d6ed61310f8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:56.758947 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11061591-3861-4e82-bd0a-d6ed61310f8f admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:56.771071 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 798/1577] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:26:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:26:56.771923 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11061591-3861-4e82-bd0a-d6ed61310f8f admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:56.782886 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11061591-3861-4e82-bd0a-d6ed61310f8f admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:56.788711 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9667cbf1-023a-41bb-b7ca-8f054c5762a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:56.789487 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667cbf1-023a-41bb-b7ca-8f054c5762a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:56.790651 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667cbf1-023a-41bb-b7ca-8f054c5762a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:56.790651 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667cbf1-023a-41bb-b7ca-8f054c5762a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:56.791265 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9667cbf1-023a-41bb-b7ca-8f054c5762a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:56.792884 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9667cbf1-023a-41bb-b7ca-8f054c5762a9 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:56.801496 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07585caa-bd74-4052-8b78-d774f213a3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:56.802186 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07585caa-bd74-4052-8b78-d774f213a3d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:56.802472 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07585caa-bd74-4052-8b78-d774f213a3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:56.802755 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07585caa-bd74-4052-8b78-d774f213a3d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:56.803115 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 799/1578] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:26:56.803454 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07585caa-bd74-4052-8b78-d774f213a3d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:56.803900 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9667cbf1-023a-41bb-b7ca-8f054c5762a9 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:56.806127 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07585caa-bd74-4052-8b78-d774f213a3d1 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:56.816910 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9667cbf1-023a-41bb-b7ca-8f054c5762a9 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:56.822196 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2dc480-d3b6-43bd-9760-b2d5a8e9f869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:56.822845 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2dc480-d3b6-43bd-9760-b2d5a8e9f869 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:56.823128 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2dc480-d3b6-43bd-9760-b2d5a8e9f869 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:56.823402 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2dc480-d3b6-43bd-9760-b2d5a8e9f869 admin admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:56.824086 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2dc480-d3b6-43bd-9760-b2d5a8e9f869 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:56.826005 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2dc480-d3b6-43bd-9760-b2d5a8e9f869 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:56.939269 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 780/1579] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:26:56 2022] POST /identity/v3/users => generated 289 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:26:56.940152 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07585caa-bd74-4052-8b78-d774f213a3d1 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:56.950268 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07585caa-bd74-4052-8b78-d774f213a3d1 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:56.955057 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b68339-a156-4d1a-93ed-093d8023b7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:56.955720 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b68339-a156-4d1a-93ed-093d8023b7cb admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:56.956004 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b68339-a156-4d1a-93ed-093d8023b7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:56.956286 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b68339-a156-4d1a-93ed-093d8023b7cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:56.957052 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b68339-a156-4d1a-93ed-093d8023b7cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:56.958658 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b68339-a156-4d1a-93ed-093d8023b7cb admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:56.969004 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 781/1580] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:26:56.969790 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b68339-a156-4d1a-93ed-093d8023b7cb admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:56.979999 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b68339-a156-4d1a-93ed-093d8023b7cb admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:56.984963 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3229e0-f523-48ad-82b7-79f355d4d81d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:56.990512 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3229e0-f523-48ad-82b7-79f355d4d81d admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:56.990512 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3229e0-f523-48ad-82b7-79f355d4d81d admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:56.990512 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3229e0-f523-48ad-82b7-79f355d4d81d admin admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:56.990512 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3229e0-f523-48ad-82b7-79f355d4d81d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:56.990512 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3229e0-f523-48ad-82b7-79f355d4d81d admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:56.997634 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:26:57.027522 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 800/1581] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:26:56 2022] POST /identity/v3/projects => generated 385 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:26:57.028366 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2dc480-d3b6-43bd-9760-b2d5a8e9f869 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:57.091198 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2dc480-d3b6-43bd-9760-b2d5a8e9f869 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:57.095729 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d2da73-de46-4415-9a82-f50e8524681e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:57.096480 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d2da73-de46-4415-9a82-f50e8524681e admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:57.096696 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d2da73-de46-4415-9a82-f50e8524681e admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:57.096966 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d2da73-de46-4415-9a82-f50e8524681e admin admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:57.097693 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d2da73-de46-4415-9a82-f50e8524681e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:57.099881 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d2da73-de46-4415-9a82-f50e8524681e admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:57.105433 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 782/1582] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:26:56 2022] POST /identity/v3/users => generated 297 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:26:57.106903 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b48f54-bfc2-4d3f-8069-ce8ebaa542c0 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:57.109224 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b48f54-bfc2-4d3f-8069-ce8ebaa542c0 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:57.109224 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b48f54-bfc2-4d3f-8069-ce8ebaa542c0 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:57.109224 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 783/1583] 10.222.0.23 () {54 vars in 961 bytes} [Tue Jun 21 18:26:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:26:57.109824 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b48f54-bfc2-4d3f-8069-ce8ebaa542c0 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:57.123529 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b48f54-bfc2-4d3f-8069-ce8ebaa542c0 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:57.128729 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee2d7e-dbd5-4729-8aac-d72129ea6a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:57.129471 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee2d7e-dbd5-4729-8aac-d72129ea6a37 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:57.129752 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee2d7e-dbd5-4729-8aac-d72129ea6a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:57.130026 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee2d7e-dbd5-4729-8aac-d72129ea6a37 admin admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:57.130706 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ee2d7e-dbd5-4729-8aac-d72129ea6a37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:57.132663 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ee2d7e-dbd5-4729-8aac-d72129ea6a37 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:57.186462 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 801/1584] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:26:57 2022] POST /identity/v3/users => generated 302 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:26:57.187427 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d2da73-de46-4415-9a82-f50e8524681e admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:57.198066 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d2da73-de46-4415-9a82-f50e8524681e admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:57.203043 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1106b6a0-7ec7-4d44-b46a-4ec921d07d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:57.203996 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1106b6a0-7ec7-4d44-b46a-4ec921d07d97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:57.204303 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1106b6a0-7ec7-4d44-b46a-4ec921d07d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:57.204622 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1106b6a0-7ec7-4d44-b46a-4ec921d07d97 admin admin] PATH_INFO: `/v3/projects/aa9cddbd0d974e7999c1c0050b0970b4/users/71742e3c8d0b4b098e9cd62f64428787/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:57.219459 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1106b6a0-7ec7-4d44-b46a-4ec921d07d97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa9cddbd0d974e7999c1c0050b0970b4, user_id=71742e3c8d0b4b098e9cd62f64428787, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:57.222151 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1106b6a0-7ec7-4d44-b46a-4ec921d07d97 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:57.237622 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 784/1585] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:26:57 2022] POST /identity/v3/projects => generated 389 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:26:57.242923 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee2d7e-dbd5-4729-8aac-d72129ea6a37 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:57.288528 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 802/1586] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:26:57 2022] PUT /identity/v3/projects/aa9cddbd0d974e7999c1c0050b0970b4/users/71742e3c8d0b4b098e9cd62f64428787/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:26:57.289376 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1106b6a0-7ec7-4d44-b46a-4ec921d07d97 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:57.303045 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee2d7e-dbd5-4729-8aac-d72129ea6a37 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:57.363271 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:26:57.371401 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae13566-fd95-4cb8-b8d7-7ed94c9bfb3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:57.372116 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae13566-fd95-4cb8-b8d7-7ed94c9bfb3a admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:57.372425 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae13566-fd95-4cb8-b8d7-7ed94c9bfb3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:57.372707 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae13566-fd95-4cb8-b8d7-7ed94c9bfb3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:57.373760 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae13566-fd95-4cb8-b8d7-7ed94c9bfb3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:57.375405 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae13566-fd95-4cb8-b8d7-7ed94c9bfb3a admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:57.375729 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1106b6a0-7ec7-4d44-b46a-4ec921d07d97 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:57.380622 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6422bfad-3afb-4ce8-9a74-9aa42daf2d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:57.381291 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422bfad-3afb-4ce8-9a74-9aa42daf2d01 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:57.381577 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422bfad-3afb-4ce8-9a74-9aa42daf2d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:57.381858 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422bfad-3afb-4ce8-9a74-9aa42daf2d01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:57.382632 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6422bfad-3afb-4ce8-9a74-9aa42daf2d01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:57.384827 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6422bfad-3afb-4ce8-9a74-9aa42daf2d01 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:57.391096 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 785/1587] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:26:57 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:26:57.391096 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae13566-fd95-4cb8-b8d7-7ed94c9bfb3a admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:57.396807 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 803/1588] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:26:57 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:26:57.402828 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6422bfad-3afb-4ce8-9a74-9aa42daf2d01 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:57.403007 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae13566-fd95-4cb8-b8d7-7ed94c9bfb3a admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:57.406388 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25f1397-5c2b-4a4c-b2c9-3ea5a585d890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:57.407687 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25f1397-5c2b-4a4c-b2c9-3ea5a585d890 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:57.407976 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25f1397-5c2b-4a4c-b2c9-3ea5a585d890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:57.408458 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25f1397-5c2b-4a4c-b2c9-3ea5a585d890 admin admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:57.409187 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25f1397-5c2b-4a4c-b2c9-3ea5a585d890 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:57.410470 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6422bfad-3afb-4ce8-9a74-9aa42daf2d01 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:57.411119 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25f1397-5c2b-4a4c-b2c9-3ea5a585d890 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:57.415261 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bdd4f0-1255-4426-8003-faa1ba850b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:57.416480 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bdd4f0-1255-4426-8003-faa1ba850b31 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:57.416767 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bdd4f0-1255-4426-8003-faa1ba850b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:57.417043 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bdd4f0-1255-4426-8003-faa1ba850b31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:57.417893 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bdd4f0-1255-4426-8003-faa1ba850b31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:57.419451 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bdd4f0-1255-4426-8003-faa1ba850b31 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:57.430259 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 804/1589] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:26:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:26:57.432138 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f44d7e-c271-498b-bbd7-6d1f40c04f11 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:57.432462 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f44d7e-c271-498b-bbd7-6d1f40c04f11 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:57.432745 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f44d7e-c271-498b-bbd7-6d1f40c04f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:57.450722 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-68f44d7e-c271-498b-bbd7-6d1f40c04f11 None None] MFA Rules not processed for user `904255dd3c1f406eb10f6fc65f9cd098`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:26:57.451915 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-68f44d7e-c271-498b-bbd7-6d1f40c04f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:26:57.520934 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 805/1590] 10.222.0.23 () {58 vars in 1058 bytes} [Tue Jun 21 18:26:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:26:57.521801 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f44d7e-c271-498b-bbd7-6d1f40c04f11 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:57.533049 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 786/1591] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:26:57 2022] POST /identity/v3/users => generated 293 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:26:57.533049 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25f1397-5c2b-4a4c-b2c9-3ea5a585d890 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:57.533576 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f44d7e-c271-498b-bbd7-6d1f40c04f11 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:57.538174 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7ae306-610d-4c5c-86fc-7f232687a502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:57.540522 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ae306-610d-4c5c-86fc-7f232687a502 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:57.540522 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ae306-610d-4c5c-86fc-7f232687a502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:57.540522 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ae306-610d-4c5c-86fc-7f232687a502 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:57.540522 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7ae306-610d-4c5c-86fc-7f232687a502 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:57.541125 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25f1397-5c2b-4a4c-b2c9-3ea5a585d890 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:57.542553 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7ae306-610d-4c5c-86fc-7f232687a502 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:57.546466 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fd1c7f-fb8c-4239-bd9a-fdd6700abedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:57.547135 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd1c7f-fb8c-4239-bd9a-fdd6700abedc admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:57.547415 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd1c7f-fb8c-4239-bd9a-fdd6700abedc admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:57.547687 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd1c7f-fb8c-4239-bd9a-fdd6700abedc admin admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:57.548423 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fd1c7f-fb8c-4239-bd9a-fdd6700abedc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:57.550325 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fd1c7f-fb8c-4239-bd9a-fdd6700abedc admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:57.552758 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 806/1592] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:26:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:26:57.557205 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90005de5-7cdf-447a-8c77-83df22713b83 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:57.557500 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90005de5-7cdf-447a-8c77-83df22713b83 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:57.557781 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90005de5-7cdf-447a-8c77-83df22713b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:57.583164 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-90005de5-7cdf-447a-8c77-83df22713b83 None None] MFA Rules not processed for user `71742e3c8d0b4b098e9cd62f64428787`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:26:57.584369 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-90005de5-7cdf-447a-8c77-83df22713b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:26:57.625885 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 787/1593] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:26:57 2022] POST /identity/v3/users => generated 295 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:26:57.627004 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fd1c7f-fb8c-4239-bd9a-fdd6700abedc admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:57.637205 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fd1c7f-fb8c-4239-bd9a-fdd6700abedc admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:57.641845 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6240c0-b717-4351-8f2f-531344c09157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:57.642510 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6240c0-b717-4351-8f2f-531344c09157 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:57.642796 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6240c0-b717-4351-8f2f-531344c09157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:57.643104 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6240c0-b717-4351-8f2f-531344c09157 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:57.643851 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6240c0-b717-4351-8f2f-531344c09157 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:57.645486 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6240c0-b717-4351-8f2f-531344c09157 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:57.656232 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 788/1594] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:26:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:26:57.657090 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6240c0-b717-4351-8f2f-531344c09157 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:57.661899 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 807/1595] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:26:57 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:26:57.661899 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90005de5-7cdf-447a-8c77-83df22713b83 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:57.670931 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6240c0-b717-4351-8f2f-531344c09157 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:57.671248 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90005de5-7cdf-447a-8c77-83df22713b83 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:57.676264 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df23da6-23aa-41c1-bcd0-2ed342314174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:57.677259 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df23da6-23aa-41c1-bcd0-2ed342314174 admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:57.677542 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df23da6-23aa-41c1-bcd0-2ed342314174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:57.677862 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df23da6-23aa-41c1-bcd0-2ed342314174 admin admin] PATH_INFO: `/v3/projects/ab91aced272948f4b6bd47ffd354162e/users/4bd6f8a502984d09b620673a33cc7dae/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:57.677964 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e08d62-d487-4437-acc3-1c60545c246a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:57.678623 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e08d62-d487-4437-acc3-1c60545c246a admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:57.678913 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e08d62-d487-4437-acc3-1c60545c246a admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:57.679205 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e08d62-d487-4437-acc3-1c60545c246a admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:57.679920 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e08d62-d487-4437-acc3-1c60545c246a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:57.682064 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e08d62-d487-4437-acc3-1c60545c246a admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:57.692207 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 808/1596] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:26:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:26:57.693415 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df23da6-23aa-41c1-bcd0-2ed342314174 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab91aced272948f4b6bd47ffd354162e, user_id=4bd6f8a502984d09b620673a33cc7dae, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:57.694462 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: WARNING py.warnings [None req-9df23da6-23aa-41c1-bcd0-2ed342314174 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 21 18:26:57.694462 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: warnings.warn(msg) Jun 21 18:26:57.694462 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: Jun 21 18:26:57.696256 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e08d62-d487-4437-acc3-1c60545c246a admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:57.696793 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df23da6-23aa-41c1-bcd0-2ed342314174 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:57.713686 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e08d62-d487-4437-acc3-1c60545c246a admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:57.715059 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784797e3-6072-4fe9-9552-dbdd01653af9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904255dd3c1f406eb10f6fc65f9cd098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:57.715703 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784797e3-6072-4fe9-9552-dbdd01653af9 service nova] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:57.715991 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784797e3-6072-4fe9-9552-dbdd01653af9 service nova] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:57.716273 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784797e3-6072-4fe9-9552-dbdd01653af9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:57.749007 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:26:57.768117 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 789/1597] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:26:57 2022] PUT /identity/v3/projects/ab91aced272948f4b6bd47ffd354162e/users/4bd6f8a502984d09b620673a33cc7dae/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:26:57.771239 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df23da6-23aa-41c1-bcd0-2ed342314174 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:57.773440 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784797e3-6072-4fe9-9552-dbdd01653af9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:57.773722 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:INSERT=10,keystone:SELECT=90 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:26:57.775446 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784797e3-6072-4fe9-9552-dbdd01653af9 service nova] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:57.785113 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 809/1598] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:26:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:26:57.786092 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784797e3-6072-4fe9-9552-dbdd01653af9 service nova] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:57.797265 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784797e3-6072-4fe9-9552-dbdd01653af9 service nova] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:57.837546 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df23da6-23aa-41c1-bcd0-2ed342314174 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:57.842607 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6bddb4-86de-47ef-9372-2b42f6a89b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:57.843299 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6bddb4-86de-47ef-9372-2b42f6a89b12 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:57.843565 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6bddb4-86de-47ef-9372-2b42f6a89b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:57.843841 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6bddb4-86de-47ef-9372-2b42f6a89b12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:57.844593 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6bddb4-86de-47ef-9372-2b42f6a89b12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:57.846214 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6bddb4-86de-47ef-9372-2b42f6a89b12 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:57.856782 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 790/1599] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:26:57 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:26:57.860741 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6bddb4-86de-47ef-9372-2b42f6a89b12 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:57.867116 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f53f1ef-d88f-4253-9396-3aedc997f7d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:57.868544 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53f1ef-d88f-4253-9396-3aedc997f7d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:57.870000 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53f1ef-d88f-4253-9396-3aedc997f7d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:57.870294 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53f1ef-d88f-4253-9396-3aedc997f7d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:57.871221 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f53f1ef-d88f-4253-9396-3aedc997f7d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:57.871469 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6bddb4-86de-47ef-9372-2b42f6a89b12 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:57.872891 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f53f1ef-d88f-4253-9396-3aedc997f7d3 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:57.877341 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c16390-48ff-4032-837e-ec6b1c3ef3d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:57.878068 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c16390-48ff-4032-837e-ec6b1c3ef3d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:57.878442 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c16390-48ff-4032-837e-ec6b1c3ef3d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:57.878756 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c16390-48ff-4032-837e-ec6b1c3ef3d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:57.879546 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c16390-48ff-4032-837e-ec6b1c3ef3d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:57.885112 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c16390-48ff-4032-837e-ec6b1c3ef3d3 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:57.885425 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 810/1600] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:26:57 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:26:57.889534 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f53f1ef-d88f-4253-9396-3aedc997f7d3 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:57.898136 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 791/1601] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:26:57 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:26:57.898136 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c16390-48ff-4032-837e-ec6b1c3ef3d3 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:57.904413 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f53f1ef-d88f-4253-9396-3aedc997f7d3 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:57.909266 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b0d63a-561b-4c15-900a-4a9ac1d52841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:57.910313 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b0d63a-561b-4c15-900a-4a9ac1d52841 admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:57.910600 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b0d63a-561b-4c15-900a-4a9ac1d52841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:57.910876 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b0d63a-561b-4c15-900a-4a9ac1d52841 admin admin] PATH_INFO: `/v3/projects/7e5e18f0b6ab45ef961c14b967941ca6/users/35d70344216248199ddf81fba88af04f/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:57.930932 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b0d63a-561b-4c15-900a-4a9ac1d52841 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e5e18f0b6ab45ef961c14b967941ca6, user_id=35d70344216248199ddf81fba88af04f, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:57.935008 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b0d63a-561b-4c15-900a-4a9ac1d52841 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:57.958851 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c16390-48ff-4032-837e-ec6b1c3ef3d3 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:57.962889 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7a4a97-dd9e-4006-b962-479c065b654d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:57.963543 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a4a97-dd9e-4006-b962-479c065b654d service cinder] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:57.963879 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a4a97-dd9e-4006-b962-479c065b654d service cinder] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:57.964102 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a4a97-dd9e-4006-b962-479c065b654d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:58.008701 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 811/1602] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:26:57 2022] PUT /identity/v3/projects/7e5e18f0b6ab45ef961c14b967941ca6/users/35d70344216248199ddf81fba88af04f/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:26:58.012920 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b0d63a-561b-4c15-900a-4a9ac1d52841 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:58.027968 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7a4a97-dd9e-4006-b962-479c065b654d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:58.029582 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7a4a97-dd9e-4006-b962-479c065b654d service cinder] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:58.038903 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 792/1603] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:26:57 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:26:58.040165 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458e3424-3528-44f2-ab0a-8ab03abfa60e None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:58.040482 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458e3424-3528-44f2-ab0a-8ab03abfa60e None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:58.041411 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458e3424-3528-44f2-ab0a-8ab03abfa60e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:58.059886 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-458e3424-3528-44f2-ab0a-8ab03abfa60e None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:26:58.061438 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-458e3424-3528-44f2-ab0a-8ab03abfa60e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:26:58.078028 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b0d63a-561b-4c15-900a-4a9ac1d52841 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:58.082768 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77d2421-54f4-4287-af03-73424d08fe70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:58.083659 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d2421-54f4-4287-af03-73424d08fe70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:58.083943 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d2421-54f4-4287-af03-73424d08fe70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:58.084221 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d2421-54f4-4287-af03-73424d08fe70 admin admin] PATH_INFO: `/v3/projects/10aedc59673c405096de01eaf61ecd6f/users/59f011e91b684e1288dad5f0ac733c0f/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:58.100062 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77d2421-54f4-4287-af03-73424d08fe70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10aedc59673c405096de01eaf61ecd6f, user_id=59f011e91b684e1288dad5f0ac733c0f, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:58.102841 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77d2421-54f4-4287-af03-73424d08fe70 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:58.107641 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:26:58.130984 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 793/1604] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:26:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:26:58.134474 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458e3424-3528-44f2-ab0a-8ab03abfa60e None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:58.141795 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458e3424-3528-44f2-ab0a-8ab03abfa60e None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:58.146609 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d435439-1fdf-4373-bbc6-e6a6cb116f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:58.147360 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d435439-1fdf-4373-bbc6-e6a6cb116f3a admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:58.147708 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d435439-1fdf-4373-bbc6-e6a6cb116f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:58.148059 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d435439-1fdf-4373-bbc6-e6a6cb116f3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:58.148926 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d435439-1fdf-4373-bbc6-e6a6cb116f3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:58.152693 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d435439-1fdf-4373-bbc6-e6a6cb116f3a admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:58.163654 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 794/1605] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:26:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:26:58.164584 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d435439-1fdf-4373-bbc6-e6a6cb116f3a admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:58.177603 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d435439-1fdf-4373-bbc6-e6a6cb116f3a admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:58.183048 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d001a044-597b-4393-a949-d83afad3e108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:58.183966 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d001a044-597b-4393-a949-d83afad3e108 admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:58.184255 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d001a044-597b-4393-a949-d83afad3e108 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:58.184588 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d001a044-597b-4393-a949-d83afad3e108 admin admin] PATH_INFO: `/v3/projects/e1dbb5859d11434195b33b48a9087b6b/users/dd21569366fc48fba4eef8cb87d24930/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:58.186926 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 812/1606] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:26:58 2022] PUT /identity/v3/projects/10aedc59673c405096de01eaf61ecd6f/users/59f011e91b684e1288dad5f0ac733c0f/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:26:58.188149 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77d2421-54f4-4287-af03-73424d08fe70 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:58.203409 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d001a044-597b-4393-a949-d83afad3e108 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1dbb5859d11434195b33b48a9087b6b, user_id=dd21569366fc48fba4eef8cb87d24930, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:58.206144 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d001a044-597b-4393-a949-d83afad3e108 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:58.228864 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:26:58.258062 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77d2421-54f4-4287-af03-73424d08fe70 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:58.263311 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f9be4d-81cb-419a-844e-c61b2763a9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:58.264019 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f9be4d-81cb-419a-844e-c61b2763a9ff admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:58.264300 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f9be4d-81cb-419a-844e-c61b2763a9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:58.264602 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f9be4d-81cb-419a-844e-c61b2763a9ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:58.265334 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f9be4d-81cb-419a-844e-c61b2763a9ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:58.266945 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f9be4d-81cb-419a-844e-c61b2763a9ff admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:58.277928 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 813/1607] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:26:58 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:26:58.279209 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408012d-2441-41af-a835-3a8f8c6c6435 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:58.279496 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408012d-2441-41af-a835-3a8f8c6c6435 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:58.279770 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408012d-2441-41af-a835-3a8f8c6c6435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:58.286669 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 795/1608] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:26:58 2022] PUT /identity/v3/projects/e1dbb5859d11434195b33b48a9087b6b/users/dd21569366fc48fba4eef8cb87d24930/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:26:58.288412 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a214b3-2d53-4e2f-a433-cb56687d0747 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:58.288718 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a214b3-2d53-4e2f-a433-cb56687d0747 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:58.289008 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a214b3-2d53-4e2f-a433-cb56687d0747 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:58.291833 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 796/1609] 10.222.0.23 () {54 vars in 912 bytes} [Tue Jun 21 18:26:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:26:58.296927 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a214b3-2d53-4e2f-a433-cb56687d0747 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:58.308072 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-5408012d-2441-41af-a835-3a8f8c6c6435 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:26:58.309496 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-5408012d-2441-41af-a835-3a8f8c6c6435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:26:58.355908 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a214b3-2d53-4e2f-a433-cb56687d0747 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:58.359644 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c85b95-fc2e-4f24-aea6-726c50269281 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:58.360301 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c85b95-fc2e-4f24-aea6-726c50269281 service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:58.360752 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c85b95-fc2e-4f24-aea6-726c50269281 service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:58.361032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c85b95-fc2e-4f24-aea6-726c50269281 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:58.368560 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 814/1610] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:26:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:26:58.370247 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5408012d-2441-41af-a835-3a8f8c6c6435 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:58.380753 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5408012d-2441-41af-a835-3a8f8c6c6435 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:58.385798 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6b69b9-c2a8-49b5-ad81-31c2ed2ea71a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:58.386781 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b69b9-c2a8-49b5-ad81-31c2ed2ea71a admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:58.387063 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b69b9-c2a8-49b5-ad81-31c2ed2ea71a admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:58.387342 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b69b9-c2a8-49b5-ad81-31c2ed2ea71a admin admin] PATH_INFO: `/v3/projects/48dc636c17544593aaf2a2409c13e540/users/5c21b9f596da493fac827d3573833ed1/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:58.403497 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6b69b9-c2a8-49b5-ad81-31c2ed2ea71a admin admin] RBAC: Authorizing `identity:create_grant(project_id=48dc636c17544593aaf2a2409c13e540, user_id=5c21b9f596da493fac827d3573833ed1, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:58.406223 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6b69b9-c2a8-49b5-ad81-31c2ed2ea71a admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:58.435596 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c85b95-fc2e-4f24-aea6-726c50269281 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:58.437073 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c85b95-fc2e-4f24-aea6-726c50269281 service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:58.447427 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 797/1611] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:26:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:26:58.448977 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab98b4-15c2-40dc-a159-92a4d298742c None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:58.449277 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab98b4-15c2-40dc-a159-92a4d298742c None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:58.449559 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab98b4-15c2-40dc-a159-92a4d298742c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:58.468245 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-96ab98b4-15c2-40dc-a159-92a4d298742c None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:26:58.469552 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-96ab98b4-15c2-40dc-a159-92a4d298742c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:26:58.477469 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 798/1612] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:26:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:26:58.478288 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab98b4-15c2-40dc-a159-92a4d298742c None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:58.488036 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 815/1613] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:26:58 2022] PUT /identity/v3/projects/48dc636c17544593aaf2a2409c13e540/users/5c21b9f596da493fac827d3573833ed1/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:26:58.489492 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df0264b-d86c-41ee-b256-308c19b7c2b8 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:58.489791 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df0264b-d86c-41ee-b256-308c19b7c2b8 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:58.490056 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df0264b-d86c-41ee-b256-308c19b7c2b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:58.495610 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab98b4-15c2-40dc-a159-92a4d298742c None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:58.511215 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-4df0264b-d86c-41ee-b256-308c19b7c2b8 None None] MFA Rules not processed for user `904255dd3c1f406eb10f6fc65f9cd098`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:26:58.512626 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-4df0264b-d86c-41ee-b256-308c19b7c2b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:26:58.549984 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:26:58.561484 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1667f04a-fc46-4ffc-af27-c3006ec604ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:58.562517 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1667f04a-fc46-4ffc-af27-c3006ec604ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:58.562799 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1667f04a-fc46-4ffc-af27-c3006ec604ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:58.563081 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1667f04a-fc46-4ffc-af27-c3006ec604ce admin admin] PATH_INFO: `/v3/projects/227bb7e0aca249aea1d7bf2ecb2903e9/users/e03c3a73166b4bbe9fca6d18ecf59221/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:58.579018 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1667f04a-fc46-4ffc-af27-c3006ec604ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=227bb7e0aca249aea1d7bf2ecb2903e9, user_id=e03c3a73166b4bbe9fca6d18ecf59221, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:58.582042 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1667f04a-fc46-4ffc-af27-c3006ec604ce admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:58.595041 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 816/1614] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Jun 21 18:26:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:26:58.596368 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac411fe-7170-4976-bb09-6c7f2ceb259c None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:58.596829 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac411fe-7170-4976-bb09-6c7f2ceb259c None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:58.597114 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac411fe-7170-4976-bb09-6c7f2ceb259c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:58.628328 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-6ac411fe-7170-4976-bb09-6c7f2ceb259c None None] MFA Rules not processed for user `dd21569366fc48fba4eef8cb87d24930`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:26:58.629653 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-6ac411fe-7170-4976-bb09-6c7f2ceb259c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:26:58.651165 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:26:58.694229 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 817/1615] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:26:58 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:26:58.695076 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac411fe-7170-4976-bb09-6c7f2ceb259c None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:58.711182 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 799/1616] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:26:58 2022] PUT /identity/v3/projects/227bb7e0aca249aea1d7bf2ecb2903e9/users/e03c3a73166b4bbe9fca6d18ecf59221/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:26:58.712428 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06121236-64df-4edb-85fb-0c163476a6db None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:58.713222 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06121236-64df-4edb-85fb-0c163476a6db None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:58.713499 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06121236-64df-4edb-85fb-0c163476a6db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:58.733285 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-06121236-64df-4edb-85fb-0c163476a6db None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:26:58.734408 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-06121236-64df-4edb-85fb-0c163476a6db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:26:58.750981 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac411fe-7170-4976-bb09-6c7f2ceb259c None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:58.803710 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ed2ad9-c5d6-43f1-977b-486c39ffc9e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:58.804511 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed2ad9-c5d6-43f1-977b-486c39ffc9e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:58.804727 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 800/1617] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:26:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:26:58.804865 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed2ad9-c5d6-43f1-977b-486c39ffc9e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:58.805132 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed2ad9-c5d6-43f1-977b-486c39ffc9e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:58.805664 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06121236-64df-4edb-85fb-0c163476a6db None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:58.816784 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ed2ad9-c5d6-43f1-977b-486c39ffc9e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:58.817922 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06121236-64df-4edb-85fb-0c163476a6db None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:58.818294 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ed2ad9-c5d6-43f1-977b-486c39ffc9e5 service neutron] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:58.823645 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043f318c-a75d-469e-a93f-63e0fc63a9ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:58.824869 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f318c-a75d-469e-a93f-63e0fc63a9ba service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:58.825144 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f318c-a75d-469e-a93f-63e0fc63a9ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:58.825479 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f318c-a75d-469e-a93f-63e0fc63a9ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:58.830948 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 818/1618] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:26:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:26:58.831914 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ed2ad9-c5d6-43f1-977b-486c39ffc9e5 service neutron] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:58.841695 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043f318c-a75d-469e-a93f-63e0fc63a9ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:58.843654 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043f318c-a75d-469e-a93f-63e0fc63a9ba service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:58.862273 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 801/1619] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:26:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:26:58.863126 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043f318c-a75d-469e-a93f-63e0fc63a9ba service neutron] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:58.873727 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043f318c-a75d-469e-a93f-63e0fc63a9ba service neutron] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:58.878976 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcda3c6-8f72-4348-818f-c2018870721c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:58.879907 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcda3c6-8f72-4348-818f-c2018870721c admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:58.880230 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcda3c6-8f72-4348-818f-c2018870721c admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:58.880561 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcda3c6-8f72-4348-818f-c2018870721c admin admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:58.881249 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcda3c6-8f72-4348-818f-c2018870721c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:58.888458 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcda3c6-8f72-4348-818f-c2018870721c admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:58.909675 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ed2ad9-c5d6-43f1-977b-486c39ffc9e5 service neutron] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:58.913338 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7930263a-94cb-4cff-9eb4-02c1f657df92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:58.913995 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7930263a-94cb-4cff-9eb4-02c1f657df92 service cinder] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:58.914277 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7930263a-94cb-4cff-9eb4-02c1f657df92 service cinder] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:58.914554 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7930263a-94cb-4cff-9eb4-02c1f657df92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:58.953579 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:26:59.002528 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7930263a-94cb-4cff-9eb4-02c1f657df92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:59.004359 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7930263a-94cb-4cff-9eb4-02c1f657df92 service cinder] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:59.018624 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 819/1620] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:26:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:26:59.018756 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 802/1621] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:26:58 2022] POST /identity/v3/projects => generated 403 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:26:59.018756 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcda3c6-8f72-4348-818f-c2018870721c admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:59.019269 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f2f5f1-ef99-4346-875b-2fd12592e9fd None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:59.019269 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f2f5f1-ef99-4346-875b-2fd12592e9fd None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:59.019420 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f2f5f1-ef99-4346-875b-2fd12592e9fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:59.022708 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-0bcda3c6-8f72-4348-818f-c2018870721c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:26:59.039772 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-54f2f5f1-ef99-4346-875b-2fd12592e9fd None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:26:59.041460 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-54f2f5f1-ef99-4346-875b-2fd12592e9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:26:59.129080 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 820/1622] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:26:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:26:59.129545 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f2f5f1-ef99-4346-875b-2fd12592e9fd None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:59.133374 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcda3c6-8f72-4348-818f-c2018870721c admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:59.133944 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ef5f99-8066-48a0-ac3e-e1d0318b8bed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:59.134267 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ef5f99-8066-48a0-ac3e-e1d0318b8bed service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:59.135286 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ef5f99-8066-48a0-ac3e-e1d0318b8bed service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:59.135707 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ef5f99-8066-48a0-ac3e-e1d0318b8bed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:59.141134 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f2f5f1-ef99-4346-875b-2fd12592e9fd None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:59.148974 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e1f3ce-1718-4a30-af3c-2d3112a1b688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:59.154336 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ef5f99-8066-48a0-ac3e-e1d0318b8bed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:59.154675 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e1f3ce-1718-4a30-af3c-2d3112a1b688 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:59.156307 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e1f3ce-1718-4a30-af3c-2d3112a1b688 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:59.156307 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e1f3ce-1718-4a30-af3c-2d3112a1b688 admin admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:59.156307 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e1f3ce-1718-4a30-af3c-2d3112a1b688 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:59.156610 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ef5f99-8066-48a0-ac3e-e1d0318b8bed service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:59.167109 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e1f3ce-1718-4a30-af3c-2d3112a1b688 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:59.179283 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 803/1623] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:26:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:26:59.182865 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ef5f99-8066-48a0-ac3e-e1d0318b8bed service neutron] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:59.202998 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ef5f99-8066-48a0-ac3e-e1d0318b8bed service neutron] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:59.217212 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fc1392-70b9-4160-af0c-3a1454c880eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:59.217893 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fc1392-70b9-4160-af0c-3a1454c880eb service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:59.218293 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fc1392-70b9-4160-af0c-3a1454c880eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:59.218511 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fc1392-70b9-4160-af0c-3a1454c880eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:59.236193 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fc1392-70b9-4160-af0c-3a1454c880eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:59.243079 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fc1392-70b9-4160-af0c-3a1454c880eb service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:59.250721 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 804/1624] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:26:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:26:59.302673 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fc1392-70b9-4160-af0c-3a1454c880eb service neutron] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:59.303332 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 821/1625] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:26:59 2022] POST /identity/v3/users => generated 308 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:26:59.314313 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fc1392-70b9-4160-af0c-3a1454c880eb service neutron] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:59.319769 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050cd877-ab40-49ce-8294-8b7c71694477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:59.322298 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050cd877-ab40-49ce-8294-8b7c71694477 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:59.322298 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050cd877-ab40-49ce-8294-8b7c71694477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:59.322298 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050cd877-ab40-49ce-8294-8b7c71694477 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:59.322298 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050cd877-ab40-49ce-8294-8b7c71694477 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:59.324321 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050cd877-ab40-49ce-8294-8b7c71694477 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:59.342078 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e1f3ce-1718-4a30-af3c-2d3112a1b688 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:59.349757 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 805/1626] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:26:59.354987 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e1f3ce-1718-4a30-af3c-2d3112a1b688 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:59.360979 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a289df8-b3d5-45be-bb34-dff473241c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:59.362170 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a289df8-b3d5-45be-bb34-dff473241c50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:59.362566 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a289df8-b3d5-45be-bb34-dff473241c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:59.362932 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a289df8-b3d5-45be-bb34-dff473241c50 admin admin] PATH_INFO: `/v3/projects/cc7ba4c219fa4c9aad2ceef396b8e80d/users/46d188322aad4529a41359f27e0356d6/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:59.389444 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a289df8-b3d5-45be-bb34-dff473241c50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc7ba4c219fa4c9aad2ceef396b8e80d, user_id=46d188322aad4529a41359f27e0356d6, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:59.394900 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a289df8-b3d5-45be-bb34-dff473241c50 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:59.473651 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 822/1627] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:26:59 2022] PUT /identity/v3/projects/cc7ba4c219fa4c9aad2ceef396b8e80d/users/46d188322aad4529a41359f27e0356d6/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:26:59.476422 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050cd877-ab40-49ce-8294-8b7c71694477 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:59.495163 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050cd877-ab40-49ce-8294-8b7c71694477 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:59.524440 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:26:59.591646 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d7a22e-97a4-4aee-a937-54f47b743117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:59.592318 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d7a22e-97a4-4aee-a937-54f47b743117 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:59.592819 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d7a22e-97a4-4aee-a937-54f47b743117 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:59.593110 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d7a22e-97a4-4aee-a937-54f47b743117 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:59.593859 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d7a22e-97a4-4aee-a937-54f47b743117 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:59.596016 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d7a22e-97a4-4aee-a937-54f47b743117 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:59.612579 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 806/1628] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:26:59.614357 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a289df8-b3d5-45be-bb34-dff473241c50 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:59.628876 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a289df8-b3d5-45be-bb34-dff473241c50 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:59.634054 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c1b6f9-e6df-48f8-af36-6afb7cfe9e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:59.636407 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1b6f9-e6df-48f8-af36-6afb7cfe9e4d admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:59.636407 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1b6f9-e6df-48f8-af36-6afb7cfe9e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:59.636407 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1b6f9-e6df-48f8-af36-6afb7cfe9e4d admin admin] PATH_INFO: `/v3/projects/cc7ba4c219fa4c9aad2ceef396b8e80d/users/46d188322aad4529a41359f27e0356d6/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:59.639473 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c1b6f9-e6df-48f8-af36-6afb7cfe9e4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc7ba4c219fa4c9aad2ceef396b8e80d, user_id=46d188322aad4529a41359f27e0356d6, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:59.642533 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c1b6f9-e6df-48f8-af36-6afb7cfe9e4d admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:26:59.685220 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 823/1629] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:26:59 2022] PUT /identity/v3/projects/cc7ba4c219fa4c9aad2ceef396b8e80d/users/46d188322aad4529a41359f27e0356d6/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:26:59.697136 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73926bd-e28d-4e5f-b581-6f97ddf20ed5 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:59.697787 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73926bd-e28d-4e5f-b581-6f97ddf20ed5 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:59.698281 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73926bd-e28d-4e5f-b581-6f97ddf20ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:59.762233 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-d73926bd-e28d-4e5f-b581-6f97ddf20ed5 None None] MFA Rules not processed for user `46d188322aad4529a41359f27e0356d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:26:59.764691 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-d73926bd-e28d-4e5f-b581-6f97ddf20ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:26:59.856962 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 807/1630] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:26:59 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:26:59.870546 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c1b6f9-e6df-48f8-af36-6afb7cfe9e4d admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:26:59.958590 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c1b6f9-e6df-48f8-af36-6afb7cfe9e4d admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:26:59.969955 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc25735-8737-4ea1-a3f2-e4777fcbf057 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:26:59.970676 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc25735-8737-4ea1-a3f2-e4777fcbf057 service cinder] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:26:59.971061 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc25735-8737-4ea1-a3f2-e4777fcbf057 service cinder] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:26:59.971413 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc25735-8737-4ea1-a3f2-e4777fcbf057 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:26:59.987399 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc25735-8737-4ea1-a3f2-e4777fcbf057 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:26:59.989279 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc25735-8737-4ea1-a3f2-e4777fcbf057 service cinder] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:00.003586 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 824/1631] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:26:59 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:27:00.376127 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73926bd-e28d-4e5f-b581-6f97ddf20ed5 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:00.547428 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73926bd-e28d-4e5f-b581-6f97ddf20ed5 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:00.554284 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d225197-19ec-415f-8998-db0d048574ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:00.555875 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d225197-19ec-415f-8998-db0d048574ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:00.556533 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d225197-19ec-415f-8998-db0d048574ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:00.556918 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d225197-19ec-415f-8998-db0d048574ad admin admin] PATH_INFO: `/v3/users/dd21569366fc48fba4eef8cb87d24930` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:00.558499 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d225197-19ec-415f-8998-db0d048574ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd21569366fc48fba4eef8cb87d24930)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:00.560965 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: WARNING py.warnings [None req-4d225197-19ec-415f-8998-db0d048574ad admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 21 18:27:00.560965 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: warnings.warn(msg) Jun 21 18:27:00.560965 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: Jun 21 18:27:00.562658 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d225197-19ec-415f-8998-db0d048574ad admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:00.623547 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ebef65-46a7-44c7-a759-49036416bec9 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:00.624832 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ebef65-46a7-44c7-a759-49036416bec9 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:00.625257 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ebef65-46a7-44c7-a759-49036416bec9 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:00.628514 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 825/1632] 10.222.0.23 () {54 vars in 961 bytes} [Tue Jun 21 18:27:00 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:27:00.640905 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76efd9-95b8-4de1-8e78-e91de5410675 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:00.641768 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76efd9-95b8-4de1-8e78-e91de5410675 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:00.642250 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76efd9-95b8-4de1-8e78-e91de5410675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:00.679881 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-3e76efd9-95b8-4de1-8e78-e91de5410675 None None] MFA Rules not processed for user `904255dd3c1f406eb10f6fc65f9cd098`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:00.682549 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-3e76efd9-95b8-4de1-8e78-e91de5410675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:00.693412 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:00.772050 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 826/1633] 10.222.0.23 () {58 vars in 1058 bytes} [Tue Jun 21 18:27:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:27:00.778549 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8427427b-0b98-4fe0-b14d-02917200222c None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:00.779023 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8427427b-0b98-4fe0-b14d-02917200222c None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:00.779390 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8427427b-0b98-4fe0-b14d-02917200222c None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:00.781204 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 827/1634] 10.222.0.23 () {54 vars in 912 bytes} [Tue Jun 21 18:27:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:27:00.786972 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde10d95-cf8b-44db-a2b5-bb9b3970a208 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:00.787315 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde10d95-cf8b-44db-a2b5-bb9b3970a208 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:00.787632 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde10d95-cf8b-44db-a2b5-bb9b3970a208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:00.810180 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-cde10d95-cf8b-44db-a2b5-bb9b3970a208 None None] MFA Rules not processed for user `904255dd3c1f406eb10f6fc65f9cd098`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:00.810475 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-cde10d95-cf8b-44db-a2b5-bb9b3970a208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:00.823410 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 828/1635] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Jun 21 18:27:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:27:00.838277 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde10d95-cf8b-44db-a2b5-bb9b3970a208 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:00.846943 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde10d95-cf8b-44db-a2b5-bb9b3970a208 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:00.852756 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed678f9-f6e5-41db-be54-77dc2ab1e13b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:00.855247 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed678f9-f6e5-41db-be54-77dc2ab1e13b service cinder] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:00.855687 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed678f9-f6e5-41db-be54-77dc2ab1e13b service cinder] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:00.856914 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed678f9-f6e5-41db-be54-77dc2ab1e13b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:00.892117 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:00.904349 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed678f9-f6e5-41db-be54-77dc2ab1e13b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:00.907074 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed678f9-f6e5-41db-be54-77dc2ab1e13b service cinder] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:00.911010 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-4d225197-19ec-415f-8998-db0d048574ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd21569366fc48fba4eef8cb87d24930'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:00.952242 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 829/1636] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:27:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:27:00.953741 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41229786-c6c3-4479-93a3-fd643088e20e None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:00.962758 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41229786-c6c3-4479-93a3-fd643088e20e None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:00.963651 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41229786-c6c3-4479-93a3-fd643088e20e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:00.991103 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-4d225197-19ec-415f-8998-db0d048574ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd21569366fc48fba4eef8cb87d24930'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:00.998919 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-4d225197-19ec-415f-8998-db0d048574ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd21569366fc48fba4eef8cb87d24930'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:01.027434 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-4d225197-19ec-415f-8998-db0d048574ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd21569366fc48fba4eef8cb87d24930'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:01.032785 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-4d225197-19ec-415f-8998-db0d048574ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd21569366fc48fba4eef8cb87d24930'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:01.051690 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-41229786-c6c3-4479-93a3-fd643088e20e None None] MFA Rules not processed for user `4bd6f8a502984d09b620673a33cc7dae`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:01.053562 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-41229786-c6c3-4479-93a3-fd643088e20e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:01.129359 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 808/1637] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:27:00 2022] DELETE /identity/v3/users/dd21569366fc48fba4eef8cb87d24930 => generated 0 bytes in 754 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:27:01.133630 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a258ef0-774c-4f0a-b149-843231c9ce03 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:01.134198 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a258ef0-774c-4f0a-b149-843231c9ce03 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:01.134198 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a258ef0-774c-4f0a-b149-843231c9ce03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:01.175708 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-7a258ef0-774c-4f0a-b149-843231c9ce03 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:01.177147 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-7a258ef0-774c-4f0a-b149-843231c9ce03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:01.183174 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 830/1638] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:27:00 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:27:01.185106 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41229786-c6c3-4479-93a3-fd643088e20e None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:01.190788 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-41229786-c6c3-4479-93a3-fd643088e20e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:01.214216 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41229786-c6c3-4479-93a3-fd643088e20e None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:01.340335 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 809/1639] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:27:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:27:01.356479 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90dd1b-372a-46de-b5b1-5ce1b8002439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:01.357244 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90dd1b-372a-46de-b5b1-5ce1b8002439 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:01.357536 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90dd1b-372a-46de-b5b1-5ce1b8002439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:01.357816 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90dd1b-372a-46de-b5b1-5ce1b8002439 admin admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:01.359173 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b90dd1b-372a-46de-b5b1-5ce1b8002439 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:01.361547 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a258ef0-774c-4f0a-b149-843231c9ce03 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:01.362271 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b90dd1b-372a-46de-b5b1-5ce1b8002439 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:01.363750 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-7a258ef0-774c-4f0a-b149-843231c9ce03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:01.438780 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a258ef0-774c-4f0a-b149-843231c9ce03 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:01.442654 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8620be-f159-4f10-9050-2fbdbb2de6fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:01.443413 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8620be-f159-4f10-9050-2fbdbb2de6fa service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:01.443761 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8620be-f159-4f10-9050-2fbdbb2de6fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:01.444106 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8620be-f159-4f10-9050-2fbdbb2de6fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:01.673903 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 831/1640] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:27:01 2022] POST /identity/v3/projects => generated 405 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:27:01.676100 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8620be-f159-4f10-9050-2fbdbb2de6fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:01.678524 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90dd1b-372a-46de-b5b1-5ce1b8002439 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:01.683137 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8620be-f159-4f10-9050-2fbdbb2de6fa service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:01.683137 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 810/1641] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:27:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:27:01.685148 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90dd1b-372a-46de-b5b1-5ce1b8002439 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:01.685148 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6784017-c081-4f2c-871c-45af456092e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:01.685148 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6784017-c081-4f2c-871c-45af456092e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:01.685148 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6784017-c081-4f2c-871c-45af456092e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:01.688240 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6784017-c081-4f2c-871c-45af456092e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:01.688240 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6784017-c081-4f2c-871c-45af456092e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:01.688240 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6784017-c081-4f2c-871c-45af456092e5 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:01.880531 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 832/1642] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:27:01 2022] POST /identity/v3/users => generated 309 bytes in 412 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:27:01.888968 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8620be-f159-4f10-9050-2fbdbb2de6fa service neutron] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:01.904186 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8620be-f159-4f10-9050-2fbdbb2de6fa service neutron] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:01.922611 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e031a7b9-3f73-4f7c-9e34-1b8b35d3a61b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:01.923374 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e031a7b9-3f73-4f7c-9e34-1b8b35d3a61b admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:01.923659 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e031a7b9-3f73-4f7c-9e34-1b8b35d3a61b admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:01.923988 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e031a7b9-3f73-4f7c-9e34-1b8b35d3a61b admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:01.924920 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e031a7b9-3f73-4f7c-9e34-1b8b35d3a61b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:01.929173 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e031a7b9-3f73-4f7c-9e34-1b8b35d3a61b admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:01.946837 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 811/1643] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:27:01.950964 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6784017-c081-4f2c-871c-45af456092e5 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:01.961399 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6784017-c081-4f2c-871c-45af456092e5 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:01.967095 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125c14c9-7acb-446e-a534-39a742f84ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:01.968371 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c14c9-7acb-446e-a534-39a742f84ff1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:01.968860 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c14c9-7acb-446e-a534-39a742f84ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:01.969316 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c14c9-7acb-446e-a534-39a742f84ff1 admin admin] PATH_INFO: `/v3/projects/36dbc0ca302a4c0cb5d21a2098dee8f1/users/ba16222bd1a948ad887450b8d42dde67/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:01.987142 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125c14c9-7acb-446e-a534-39a742f84ff1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36dbc0ca302a4c0cb5d21a2098dee8f1, user_id=ba16222bd1a948ad887450b8d42dde67, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:01.991031 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125c14c9-7acb-446e-a534-39a742f84ff1 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:01.999617 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:02.115717 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 833/1644] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:27:01 2022] PUT /identity/v3/projects/36dbc0ca302a4c0cb5d21a2098dee8f1/users/ba16222bd1a948ad887450b8d42dde67/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:27:02.116637 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e031a7b9-3f73-4f7c-9e34-1b8b35d3a61b admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:02.136240 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e031a7b9-3f73-4f7c-9e34-1b8b35d3a61b admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:02.301871 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11e6472-2302-4b66-b402-05ea0ea837e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:02.303856 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e6472-2302-4b66-b402-05ea0ea837e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:02.304250 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e6472-2302-4b66-b402-05ea0ea837e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:02.304831 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e6472-2302-4b66-b402-05ea0ea837e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:02.305786 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11e6472-2302-4b66-b402-05ea0ea837e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:02.308198 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11e6472-2302-4b66-b402-05ea0ea837e8 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:02.326618 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 812/1645] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:27:02 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:27:02.327858 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125c14c9-7acb-446e-a534-39a742f84ff1 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:02.339148 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125c14c9-7acb-446e-a534-39a742f84ff1 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:02.344677 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a44e0ac-fde4-494e-bace-4c123a8d1a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:02.347788 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a44e0ac-fde4-494e-bace-4c123a8d1a26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:02.348679 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a44e0ac-fde4-494e-bace-4c123a8d1a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:02.349520 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a44e0ac-fde4-494e-bace-4c123a8d1a26 admin admin] PATH_INFO: `/v3/projects/36dbc0ca302a4c0cb5d21a2098dee8f1/users/ba16222bd1a948ad887450b8d42dde67/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:02.353632 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a44e0ac-fde4-494e-bace-4c123a8d1a26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36dbc0ca302a4c0cb5d21a2098dee8f1, user_id=ba16222bd1a948ad887450b8d42dde67, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:02.358211 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a44e0ac-fde4-494e-bace-4c123a8d1a26 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:02.372004 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11e6472-2302-4b66-b402-05ea0ea837e8 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:02.374195 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-b11e6472-2302-4b66-b402-05ea0ea837e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:02.381250 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 834/1646] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:27:02 2022] PUT /identity/v3/projects/36dbc0ca302a4c0cb5d21a2098dee8f1/users/ba16222bd1a948ad887450b8d42dde67/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:27:02.392599 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:02.402069 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11e6472-2302-4b66-b402-05ea0ea837e8 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:02.541093 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c92a7fd-381f-4f68-9372-06751248d46c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:02.542950 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c92a7fd-381f-4f68-9372-06751248d46c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:02.544986 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c92a7fd-381f-4f68-9372-06751248d46c admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:02.545599 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c92a7fd-381f-4f68-9372-06751248d46c admin admin] PATH_INFO: `/v3/projects/e1dbb5859d11434195b33b48a9087b6b` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:02.547188 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c92a7fd-381f-4f68-9372-06751248d46c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1dbb5859d11434195b33b48a9087b6b)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:02.549365 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c92a7fd-381f-4f68-9372-06751248d46c admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:02.723785 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-2c92a7fd-381f-4f68-9372-06751248d46c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1dbb5859d11434195b33b48a9087b6b', 'request_id': 'req-2c92a7fd-381f-4f68-9372-06751248d46c'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:02.838098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-2c92a7fd-381f-4f68-9372-06751248d46c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1dbb5859d11434195b33b48a9087b6b', 'request_id': 'req-2c92a7fd-381f-4f68-9372-06751248d46c'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:02.852204 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-2c92a7fd-381f-4f68-9372-06751248d46c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1dbb5859d11434195b33b48a9087b6b', 'request_id': 'req-2c92a7fd-381f-4f68-9372-06751248d46c'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:02.865834 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-2c92a7fd-381f-4f68-9372-06751248d46c admin admin] The token cache is being invalidate because project e1dbb5859d11434195b33b48a9087b6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:27:02.866924 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-2c92a7fd-381f-4f68-9372-06751248d46c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:02.867819 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-2c92a7fd-381f-4f68-9372-06751248d46c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:02.870860 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 813/1647] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:27:02 2022] DELETE /identity/v3/projects/e1dbb5859d11434195b33b48a9087b6b => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:27:02.875856 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a44e0ac-fde4-494e-bace-4c123a8d1a26 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:02.878383 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-9a44e0ac-fde4-494e-bace-4c123a8d1a26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:02.898273 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a44e0ac-fde4-494e-bace-4c123a8d1a26 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:03.004659 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888dc4a7-7a77-4375-a9f4-0e32474e2ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:03.006374 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888dc4a7-7a77-4375-a9f4-0e32474e2ec5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:03.006374 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888dc4a7-7a77-4375-a9f4-0e32474e2ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:03.006374 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888dc4a7-7a77-4375-a9f4-0e32474e2ec5 admin admin] PATH_INFO: `/v3/users/46d188322aad4529a41359f27e0356d6` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:03.007706 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888dc4a7-7a77-4375-a9f4-0e32474e2ec5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46d188322aad4529a41359f27e0356d6)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:03.011099 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888dc4a7-7a77-4375-a9f4-0e32474e2ec5 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:03.294260 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-888dc4a7-7a77-4375-a9f4-0e32474e2ec5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46d188322aad4529a41359f27e0356d6'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:03.312916 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-888dc4a7-7a77-4375-a9f4-0e32474e2ec5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46d188322aad4529a41359f27e0356d6'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:03.550833 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-888dc4a7-7a77-4375-a9f4-0e32474e2ec5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46d188322aad4529a41359f27e0356d6'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:03.551440 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-888dc4a7-7a77-4375-a9f4-0e32474e2ec5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46d188322aad4529a41359f27e0356d6'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:03.577203 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-888dc4a7-7a77-4375-a9f4-0e32474e2ec5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46d188322aad4529a41359f27e0356d6'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:03.629547 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 835/1648] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:27:02 2022] DELETE /identity/v3/users/46d188322aad4529a41359f27e0356d6 => generated 0 bytes in 744 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:27:04.012354 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:04.213583 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c92a7fd-381f-4f68-9372-06751248d46c admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:04.238886 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c92a7fd-381f-4f68-9372-06751248d46c admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:04.376904 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5354d0-1f17-44cb-a49f-7d84e8c7a79d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:04.378804 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5354d0-1f17-44cb-a49f-7d84e8c7a79d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:04.379198 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5354d0-1f17-44cb-a49f-7d84e8c7a79d admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:04.379573 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5354d0-1f17-44cb-a49f-7d84e8c7a79d admin admin] PATH_INFO: `/v3/projects/cc7ba4c219fa4c9aad2ceef396b8e80d` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:04.387078 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5354d0-1f17-44cb-a49f-7d84e8c7a79d admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc7ba4c219fa4c9aad2ceef396b8e80d)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:04.393223 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5354d0-1f17-44cb-a49f-7d84e8c7a79d admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:04.989698 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-ad5354d0-1f17-44cb-a49f-7d84e8c7a79d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc7ba4c219fa4c9aad2ceef396b8e80d', 'request_id': 'req-ad5354d0-1f17-44cb-a49f-7d84e8c7a79d'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:05.094157 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-ad5354d0-1f17-44cb-a49f-7d84e8c7a79d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc7ba4c219fa4c9aad2ceef396b8e80d', 'request_id': 'req-ad5354d0-1f17-44cb-a49f-7d84e8c7a79d'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:05.102449 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-ad5354d0-1f17-44cb-a49f-7d84e8c7a79d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc7ba4c219fa4c9aad2ceef396b8e80d', 'request_id': 'req-ad5354d0-1f17-44cb-a49f-7d84e8c7a79d'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:05.125854 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-ad5354d0-1f17-44cb-a49f-7d84e8c7a79d admin admin] The token cache is being invalidate because project cc7ba4c219fa4c9aad2ceef396b8e80d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:27:05.128093 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-ad5354d0-1f17-44cb-a49f-7d84e8c7a79d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:05.128742 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-ad5354d0-1f17-44cb-a49f-7d84e8c7a79d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:05.137506 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 814/1649] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:27:04 2022] DELETE /identity/v3/projects/cc7ba4c219fa4c9aad2ceef396b8e80d => generated 0 bytes in 921 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:27:05.139042 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ed3f39-bb46-480f-9405-3c02a5344cb9 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:05.142114 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ed3f39-bb46-480f-9405-3c02a5344cb9 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:05.142553 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ed3f39-bb46-480f-9405-3c02a5344cb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:05.195336 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-94ed3f39-bb46-480f-9405-3c02a5344cb9 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:05.198890 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-94ed3f39-bb46-480f-9405-3c02a5344cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:05.311472 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:05.339996 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d5aff9-b907-49d2-99ce-59dbf4f098d8 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:05.340529 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d5aff9-b907-49d2-99ce-59dbf4f098d8 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:05.341109 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d5aff9-b907-49d2-99ce-59dbf4f098d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:05.353462 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 836/1650] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:27:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:27:05.380608 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-38d5aff9-b907-49d2-99ce-59dbf4f098d8 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:05.387567 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-38d5aff9-b907-49d2-99ce-59dbf4f098d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:05.415450 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 815/1651] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:27:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:27:05.418641 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ed3f39-bb46-480f-9405-3c02a5344cb9 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:05.462503 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ed3f39-bb46-480f-9405-3c02a5344cb9 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:05.475797 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b271c32d-62e6-4826-9c97-931e275281db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:05.476631 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b271c32d-62e6-4826-9c97-931e275281db admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:05.476926 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b271c32d-62e6-4826-9c97-931e275281db admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:05.477250 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b271c32d-62e6-4826-9c97-931e275281db admin admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:05.478552 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b271c32d-62e6-4826-9c97-931e275281db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:27:05.479607 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b271c32d-62e6-4826-9c97-931e275281db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:05.484206 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b271c32d-62e6-4826-9c97-931e275281db admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:05.516530 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 837/1652] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:27:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:27:05.517557 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d5aff9-b907-49d2-99ce-59dbf4f098d8 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:05.538685 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d5aff9-b907-49d2-99ce-59dbf4f098d8 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:05.546929 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2058791e-47f0-4645-ae61-287b2a37294a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:05.547920 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2058791e-47f0-4645-ae61-287b2a37294a admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:05.548435 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2058791e-47f0-4645-ae61-287b2a37294a admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:05.548920 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2058791e-47f0-4645-ae61-287b2a37294a admin admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:05.550007 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2058791e-47f0-4645-ae61-287b2a37294a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:05.558239 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2058791e-47f0-4645-ae61-287b2a37294a admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:05.559069 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a631a6-51f1-42b1-80eb-613bfd6293c3 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:05.559362 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a631a6-51f1-42b1-80eb-613bfd6293c3 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:05.559644 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a631a6-51f1-42b1-80eb-613bfd6293c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:05.615814 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-87a631a6-51f1-42b1-80eb-613bfd6293c3 None None] MFA Rules not processed for user `ba16222bd1a948ad887450b8d42dde67`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:05.622243 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-87a631a6-51f1-42b1-80eb-613bfd6293c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:05.756344 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 816/1653] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:27:05 2022] POST /identity/v3/projects => generated 371 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:27:05.761107 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2058791e-47f0-4645-ae61-287b2a37294a admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:05.810465 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 838/1654] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:27:05 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:27:05.957383 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2058791e-47f0-4645-ae61-287b2a37294a admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:05.979249 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9062001-67ea-4bd7-9d9f-448d8cb6bbf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:05.980431 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9062001-67ea-4bd7-9d9f-448d8cb6bbf5 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:05.980909 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9062001-67ea-4bd7-9d9f-448d8cb6bbf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:05.981382 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9062001-67ea-4bd7-9d9f-448d8cb6bbf5 admin admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:05.982418 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9062001-67ea-4bd7-9d9f-448d8cb6bbf5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:05.986351 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9062001-67ea-4bd7-9d9f-448d8cb6bbf5 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:06.097525 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 817/1655] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:27:05 2022] POST /identity/v3/users => generated 286 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:27:06.103898 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a631a6-51f1-42b1-80eb-613bfd6293c3 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:06.120343 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a631a6-51f1-42b1-80eb-613bfd6293c3 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:06.133461 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cebe70-d38c-4986-9925-4df8356e6a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:06.134133 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cebe70-d38c-4986-9925-4df8356e6a05 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:06.135070 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cebe70-d38c-4986-9925-4df8356e6a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:06.135495 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cebe70-d38c-4986-9925-4df8356e6a05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:06.139583 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cebe70-d38c-4986-9925-4df8356e6a05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:06.141739 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cebe70-d38c-4986-9925-4df8356e6a05 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:06.169215 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 839/1656] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:27:06 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:27:06.173826 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9062001-67ea-4bd7-9d9f-448d8cb6bbf5 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:06.190706 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9062001-67ea-4bd7-9d9f-448d8cb6bbf5 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:06.211004 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07fed74-9db2-459c-877a-84048910023f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:06.212580 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07fed74-9db2-459c-877a-84048910023f admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:06.213604 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07fed74-9db2-459c-877a-84048910023f admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:06.214100 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07fed74-9db2-459c-877a-84048910023f admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:06.216946 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07fed74-9db2-459c-877a-84048910023f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:06.221109 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07fed74-9db2-459c-877a-84048910023f admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:06.258341 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 818/1657] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:27:06 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:27:06.262082 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cebe70-d38c-4986-9925-4df8356e6a05 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:06.275049 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cebe70-d38c-4986-9925-4df8356e6a05 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:06.293245 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd03fe1c-d8bf-4efa-aab7-51eb7e86ccdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:06.294008 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd03fe1c-d8bf-4efa-aab7-51eb7e86ccdb admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:06.294299 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd03fe1c-d8bf-4efa-aab7-51eb7e86ccdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:06.294582 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd03fe1c-d8bf-4efa-aab7-51eb7e86ccdb admin admin] PATH_INFO: `/v3/projects/e45e6f9e8c344d7b9a21d92e27dafd9b/users/47701eb554a24d049820beb63129e16f/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:06.353423 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd03fe1c-d8bf-4efa-aab7-51eb7e86ccdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e45e6f9e8c344d7b9a21d92e27dafd9b, user_id=47701eb554a24d049820beb63129e16f, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:06.356446 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd03fe1c-d8bf-4efa-aab7-51eb7e86ccdb admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:06.459583 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 840/1658] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:27:06 2022] PUT /identity/v3/projects/e45e6f9e8c344d7b9a21d92e27dafd9b/users/47701eb554a24d049820beb63129e16f/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:27:06.475612 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ad886-68e7-4b94-83b1-e185008ad5aa None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:06.476310 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ad886-68e7-4b94-83b1-e185008ad5aa None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:06.476850 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ad886-68e7-4b94-83b1-e185008ad5aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:06.536618 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-bb9ad886-68e7-4b94-83b1-e185008ad5aa None None] MFA Rules not processed for user `47701eb554a24d049820beb63129e16f`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:06.537863 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-bb9ad886-68e7-4b94-83b1-e185008ad5aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:06.695991 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 819/1659] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:27:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:27:06.697651 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd03fe1c-d8bf-4efa-aab7-51eb7e86ccdb admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:06.731215 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:06.948336 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd03fe1c-d8bf-4efa-aab7-51eb7e86ccdb admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:06.957409 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a8aa63-de22-4501-9095-d11ead399fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:06.958164 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a8aa63-de22-4501-9095-d11ead399fc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:06.958456 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a8aa63-de22-4501-9095-d11ead399fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:06.958785 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a8aa63-de22-4501-9095-d11ead399fc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:06.959447 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a8aa63-de22-4501-9095-d11ead399fc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:06.966197 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a8aa63-de22-4501-9095-d11ead399fc7 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:07.066463 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:07.172230 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 841/1660] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:27:06 2022] POST /identity/v3/projects => generated 373 bytes in 470 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:27:07.173663 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9ad886-68e7-4b94-83b1-e185008ad5aa None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:07.196491 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9ad886-68e7-4b94-83b1-e185008ad5aa None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:07.364139 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cdf94a-fbc9-4892-8368-b5082b766da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:07.364885 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cdf94a-fbc9-4892-8368-b5082b766da1 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:07.365173 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cdf94a-fbc9-4892-8368-b5082b766da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:07.365454 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cdf94a-fbc9-4892-8368-b5082b766da1 admin admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:07.366167 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cdf94a-fbc9-4892-8368-b5082b766da1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:07.368590 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cdf94a-fbc9-4892-8368-b5082b766da1 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:07.444002 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 820/1661] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:27:07 2022] POST /identity/v3/users => generated 293 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:27:07.449520 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a8aa63-de22-4501-9095-d11ead399fc7 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:07.466489 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a8aa63-de22-4501-9095-d11ead399fc7 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:07.481082 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c3b16-d557-4f6e-a272-cf8ef1b4cf54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:07.482077 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c3b16-d557-4f6e-a272-cf8ef1b4cf54 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:07.482681 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c3b16-d557-4f6e-a272-cf8ef1b4cf54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:07.483477 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c3b16-d557-4f6e-a272-cf8ef1b4cf54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:07.487657 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807c3b16-d557-4f6e-a272-cf8ef1b4cf54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:07.489560 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807c3b16-d557-4f6e-a272-cf8ef1b4cf54 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:07.520148 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 842/1662] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:27:07 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:27:07.521045 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cdf94a-fbc9-4892-8368-b5082b766da1 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:07.536558 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cdf94a-fbc9-4892-8368-b5082b766da1 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:07.548601 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4d0c81-5afb-4d57-a7ce-1f3602ccaad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:07.550576 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d0c81-5afb-4d57-a7ce-1f3602ccaad1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:07.550865 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d0c81-5afb-4d57-a7ce-1f3602ccaad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:07.551146 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d0c81-5afb-4d57-a7ce-1f3602ccaad1 admin admin] PATH_INFO: `/v3/projects/41a79443bf5a4ebb87d013ee6c543d20/users/a97bab8260354bedb55c4b9e6c3cb57c/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:07.590687 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4d0c81-5afb-4d57-a7ce-1f3602ccaad1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41a79443bf5a4ebb87d013ee6c543d20, user_id=a97bab8260354bedb55c4b9e6c3cb57c, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:07.596660 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4d0c81-5afb-4d57-a7ce-1f3602ccaad1 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:07.730424 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 821/1663] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:27:07 2022] PUT /identity/v3/projects/41a79443bf5a4ebb87d013ee6c543d20/users/a97bab8260354bedb55c4b9e6c3cb57c/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:27:07.734426 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c3b16-d557-4f6e-a272-cf8ef1b4cf54 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:07.751196 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c3b16-d557-4f6e-a272-cf8ef1b4cf54 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:07.897459 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1aa553-83dd-4898-ac34-7639fed4c0ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:07.898634 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1aa553-83dd-4898-ac34-7639fed4c0ac admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:07.899590 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1aa553-83dd-4898-ac34-7639fed4c0ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:07.900060 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1aa553-83dd-4898-ac34-7639fed4c0ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:07.904440 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1aa553-83dd-4898-ac34-7639fed4c0ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:07.909275 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1aa553-83dd-4898-ac34-7639fed4c0ac admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:07.935771 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4d0c81-5afb-4d57-a7ce-1f3602ccaad1 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:07.948436 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4d0c81-5afb-4d57-a7ce-1f3602ccaad1 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:07.951172 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 843/1664] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:27:07 2022] GET /identity/v3/roles => generated 1420 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:27:07.966695 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b512aa2-8cde-4b42-83d8-4c170e8a0236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:07.968207 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b512aa2-8cde-4b42-83d8-4c170e8a0236 admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:07.968663 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b512aa2-8cde-4b42-83d8-4c170e8a0236 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:07.969650 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b512aa2-8cde-4b42-83d8-4c170e8a0236 admin admin] PATH_INFO: `/v3/projects/41a79443bf5a4ebb87d013ee6c543d20/users/a97bab8260354bedb55c4b9e6c3cb57c/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:07.976592 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b512aa2-8cde-4b42-83d8-4c170e8a0236 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41a79443bf5a4ebb87d013ee6c543d20, user_id=a97bab8260354bedb55c4b9e6c3cb57c, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:07.983494 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b512aa2-8cde-4b42-83d8-4c170e8a0236 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:08.008949 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 822/1665] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:27:07 2022] PUT /identity/v3/projects/41a79443bf5a4ebb87d013ee6c543d20/users/a97bab8260354bedb55c4b9e6c3cb57c/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:27:08.022875 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4044c47b-494e-4a36-bd9c-b01195982fca None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:08.023330 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4044c47b-494e-4a36-bd9c-b01195982fca None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:08.023791 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4044c47b-494e-4a36-bd9c-b01195982fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:08.073004 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-4044c47b-494e-4a36-bd9c-b01195982fca None None] MFA Rules not processed for user `a97bab8260354bedb55c4b9e6c3cb57c`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:08.073865 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-4044c47b-494e-4a36-bd9c-b01195982fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:08.338750 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 844/1666] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:27:08 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:27:08.346013 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b512aa2-8cde-4b42-83d8-4c170e8a0236 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:08.349885 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-0b512aa2-8cde-4b42-83d8-4c170e8a0236 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:08.477679 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b512aa2-8cde-4b42-83d8-4c170e8a0236 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:08.483117 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3505b10a-63a5-4a84-b9a2-f095c36fdfa9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:08.483888 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505b10a-63a5-4a84-b9a2-f095c36fdfa9 service cinder] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:08.484239 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505b10a-63a5-4a84-b9a2-f095c36fdfa9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:08.484638 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505b10a-63a5-4a84-b9a2-f095c36fdfa9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:08.503139 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3505b10a-63a5-4a84-b9a2-f095c36fdfa9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:08.509170 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3505b10a-63a5-4a84-b9a2-f095c36fdfa9 service cinder] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:08.517593 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 823/1667] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:27:08 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:27:08.750294 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf35735-2ce8-4b91-bf00-94f24cca0a0b None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:08.750885 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf35735-2ce8-4b91-bf00-94f24cca0a0b None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:08.751361 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf35735-2ce8-4b91-bf00-94f24cca0a0b None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:08.757792 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 845/1668] 10.222.0.23 () {54 vars in 921 bytes} [Tue Jun 21 18:27:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:27:08.772685 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723c8fe0-09c2-4145-a543-72c8a0552117 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:08.773667 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723c8fe0-09c2-4145-a543-72c8a0552117 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:08.774167 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723c8fe0-09c2-4145-a543-72c8a0552117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:08.817056 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-723c8fe0-09c2-4145-a543-72c8a0552117 None None] MFA Rules not processed for user `938187967efe49be9c5389d1feef81ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:08.818795 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-723c8fe0-09c2-4145-a543-72c8a0552117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:08.957481 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf35735-2ce8-4b91-bf00-94f24cca0a0b None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:08.959855 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-dcf35735-2ce8-4b91-bf00-94f24cca0a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:08.969963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:08.986685 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf35735-2ce8-4b91-bf00-94f24cca0a0b None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:08.992128 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:08.997762 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a585ad1-eb0e-4e1d-8692-166ffc132e19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:08.999073 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a585ad1-eb0e-4e1d-8692-166ffc132e19 service cinder] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:09.000926 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a585ad1-eb0e-4e1d-8692-166ffc132e19 service cinder] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:09.001792 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a585ad1-eb0e-4e1d-8692-166ffc132e19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:09.106129 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 824/1669] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Jun 21 18:27:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:27:09.118491 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c06fa3e-06da-468f-8bf3-8456096d0dcb None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:09.119386 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c06fa3e-06da-468f-8bf3-8456096d0dcb None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:09.120007 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c06fa3e-06da-468f-8bf3-8456096d0dcb None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:09.122678 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 825/1670] 10.222.0.23 () {54 vars in 960 bytes} [Tue Jun 21 18:27:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:27:09.130076 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20550fd9-3d30-4d3a-bcb1-fb4175a9a0d4 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:09.130354 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20550fd9-3d30-4d3a-bcb1-fb4175a9a0d4 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:09.133038 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20550fd9-3d30-4d3a-bcb1-fb4175a9a0d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:09.157764 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a585ad1-eb0e-4e1d-8692-166ffc132e19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:09.170944 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a585ad1-eb0e-4e1d-8692-166ffc132e19 service cinder] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:09.199799 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-20550fd9-3d30-4d3a-bcb1-fb4175a9a0d4 None None] MFA Rules not processed for user `938187967efe49be9c5389d1feef81ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:09.218402 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-20550fd9-3d30-4d3a-bcb1-fb4175a9a0d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:09.234498 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 846/1671] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:27:08 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:27:09.270792 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 826/1672] 10.222.0.23 () {58 vars in 1057 bytes} [Tue Jun 21 18:27:09 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:27:09.278493 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a585ad1-eb0e-4e1d-8692-166ffc132e19 service cinder] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:09.296946 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a585ad1-eb0e-4e1d-8692-166ffc132e19 service cinder] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:09.305208 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e13a6f1-f3e0-4467-8c9c-ef1770677fe7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938187967efe49be9c5389d1feef81ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:09.306027 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13a6f1-f3e0-4467-8c9c-ef1770677fe7 service placement] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:09.307148 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13a6f1-f3e0-4467-8c9c-ef1770677fe7 service placement] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:09.307634 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13a6f1-f3e0-4467-8c9c-ef1770677fe7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:09.323918 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e13a6f1-f3e0-4467-8c9c-ef1770677fe7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:09.328303 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e13a6f1-f3e0-4467-8c9c-ef1770677fe7 service placement] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:09.369998 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 847/1673] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:27:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:27:09.442099 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20550fd9-3d30-4d3a-bcb1-fb4175a9a0d4 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:09.444037 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-20550fd9-3d30-4d3a-bcb1-fb4175a9a0d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:09.584878 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20550fd9-3d30-4d3a-bcb1-fb4175a9a0d4 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:09.591419 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fa0ee6-3140-4152-ac52-d4d6e7655ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:09.592324 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fa0ee6-3140-4152-ac52-d4d6e7655ffc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:09.592666 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fa0ee6-3140-4152-ac52-d4d6e7655ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:09.592947 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fa0ee6-3140-4152-ac52-d4d6e7655ffc admin admin] PATH_INFO: `/v3/users/61b74b7ef05740fdad199df473316eea` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:09.594743 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fa0ee6-3140-4152-ac52-d4d6e7655ffc admin admin] RBAC: Authorizing `identity:delete_user(user_id=61b74b7ef05740fdad199df473316eea)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:09.596807 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fa0ee6-3140-4152-ac52-d4d6e7655ffc admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:10.052668 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-f5fa0ee6-3140-4152-ac52-d4d6e7655ffc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61b74b7ef05740fdad199df473316eea'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:10.098288 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-f5fa0ee6-3140-4152-ac52-d4d6e7655ffc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61b74b7ef05740fdad199df473316eea'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:10.103719 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-f5fa0ee6-3140-4152-ac52-d4d6e7655ffc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61b74b7ef05740fdad199df473316eea'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:10.122318 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-f5fa0ee6-3140-4152-ac52-d4d6e7655ffc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61b74b7ef05740fdad199df473316eea'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:10.127421 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-f5fa0ee6-3140-4152-ac52-d4d6e7655ffc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61b74b7ef05740fdad199df473316eea'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:10.225031 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 827/1674] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:27:09 2022] DELETE /identity/v3/users/61b74b7ef05740fdad199df473316eea => generated 0 bytes in 781 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:27:10.226455 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7c908-b331-47ff-8b38-2902d61e0675 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:10.226750 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7c908-b331-47ff-8b38-2902d61e0675 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:10.227204 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7c908-b331-47ff-8b38-2902d61e0675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:10.260240 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-c5e7c908-b331-47ff-8b38-2902d61e0675 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:10.261533 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-c5e7c908-b331-47ff-8b38-2902d61e0675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:10.433056 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 848/1675] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:27:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:27:10.439440 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fa0ee6-3140-4152-ac52-d4d6e7655ffc admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:10.440555 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-f5fa0ee6-3140-4152-ac52-d4d6e7655ffc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:10.641156 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fa0ee6-3140-4152-ac52-d4d6e7655ffc admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:10.655149 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2262fb0f-1cfd-49b2-9b25-3b3da1813b42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:10.655844 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2262fb0f-1cfd-49b2-9b25-3b3da1813b42 service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:10.656189 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2262fb0f-1cfd-49b2-9b25-3b3da1813b42 service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:10.656519 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2262fb0f-1cfd-49b2-9b25-3b3da1813b42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:10.670680 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:10.678002 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2262fb0f-1cfd-49b2-9b25-3b3da1813b42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:10.680786 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2262fb0f-1cfd-49b2-9b25-3b3da1813b42 service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:10.713265 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 828/1676] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:27:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:27:12.418749 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e7c908-b331-47ff-8b38-2902d61e0675 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:12.420543 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-c5e7c908-b331-47ff-8b38-2902d61e0675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:12.442409 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e7c908-b331-47ff-8b38-2902d61e0675 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:12.454969 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ab397-db85-4f73-affc-25595f4a1693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:12.454969 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ab397-db85-4f73-affc-25595f4a1693 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:12.454969 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ab397-db85-4f73-affc-25595f4a1693 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:12.454969 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ab397-db85-4f73-affc-25595f4a1693 admin admin] PATH_INFO: `/v3/users/4bd6f8a502984d09b620673a33cc7dae` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:12.455356 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8ab397-db85-4f73-affc-25595f4a1693 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bd6f8a502984d09b620673a33cc7dae)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:12.459919 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8ab397-db85-4f73-affc-25595f4a1693 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:12.748863 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-3c8ab397-db85-4f73-affc-25595f4a1693 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bd6f8a502984d09b620673a33cc7dae'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:12.816036 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-3c8ab397-db85-4f73-affc-25595f4a1693 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bd6f8a502984d09b620673a33cc7dae'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:12.902768 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2262fb0f-1cfd-49b2-9b25-3b3da1813b42 service neutron] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:12.904325 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-2262fb0f-1cfd-49b2-9b25-3b3da1813b42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:12.950123 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2262fb0f-1cfd-49b2-9b25-3b3da1813b42 service neutron] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:12.962303 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-3c8ab397-db85-4f73-affc-25595f4a1693 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bd6f8a502984d09b620673a33cc7dae'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:12.963734 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-3c8ab397-db85-4f73-affc-25595f4a1693 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bd6f8a502984d09b620673a33cc7dae'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:12.978745 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-3c8ab397-db85-4f73-affc-25595f4a1693 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bd6f8a502984d09b620673a33cc7dae'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:12.983290 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1ca589-e198-455c-876b-cc59c136a563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:12.984234 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ca589-e198-455c-876b-cc59c136a563 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:12.984546 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ca589-e198-455c-876b-cc59c136a563 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:12.984823 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ca589-e198-455c-876b-cc59c136a563 admin admin] PATH_INFO: `/v3/users/71742e3c8d0b4b098e9cd62f64428787` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:12.987233 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:13.000113 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1ca589-e198-455c-876b-cc59c136a563 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71742e3c8d0b4b098e9cd62f64428787)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:13.008212 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 849/1677] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:27:12 2022] DELETE /identity/v3/users/4bd6f8a502984d09b620673a33cc7dae => generated 0 bytes in 590 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:27:13.018050 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1ca589-e198-455c-876b-cc59c136a563 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:13.019796 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ab397-db85-4f73-affc-25595f4a1693 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:13.026558 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-3c8ab397-db85-4f73-affc-25595f4a1693 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:13.261403 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ab397-db85-4f73-affc-25595f4a1693 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:13.284989 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d290e1-2704-488c-9b37-758c2a821aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:13.286424 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d290e1-2704-488c-9b37-758c2a821aa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:13.286865 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d290e1-2704-488c-9b37-758c2a821aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:13.287286 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d290e1-2704-488c-9b37-758c2a821aa4 admin admin] PATH_INFO: `/v3/projects/d3bfd39123f64e27805ea90036c1340c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:13.289260 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d290e1-2704-488c-9b37-758c2a821aa4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3bfd39123f64e27805ea90036c1340c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:13.291133 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: WARNING py.warnings [None req-36d290e1-2704-488c-9b37-758c2a821aa4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 21 18:27:13.291133 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: warnings.warn(msg) Jun 21 18:27:13.291133 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: Jun 21 18:27:13.296934 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d290e1-2704-488c-9b37-758c2a821aa4 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:13.353723 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-1f1ca589-e198-455c-876b-cc59c136a563 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71742e3c8d0b4b098e9cd62f64428787'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:13.388367 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-1f1ca589-e198-455c-876b-cc59c136a563 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71742e3c8d0b4b098e9cd62f64428787'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:13.389556 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-1f1ca589-e198-455c-876b-cc59c136a563 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71742e3c8d0b4b098e9cd62f64428787'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:13.409131 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-1f1ca589-e198-455c-876b-cc59c136a563 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71742e3c8d0b4b098e9cd62f64428787'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:13.410123 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-1f1ca589-e198-455c-876b-cc59c136a563 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71742e3c8d0b4b098e9cd62f64428787'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:13.526630 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-36d290e1-2704-488c-9b37-758c2a821aa4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3bfd39123f64e27805ea90036c1340c', 'request_id': 'req-36d290e1-2704-488c-9b37-758c2a821aa4'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:13.527343 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 829/1678] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:27:12 2022] DELETE /identity/v3/users/71742e3c8d0b4b098e9cd62f64428787 => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:27:13.527613 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0824e-a114-48a3-8803-30e54ea43798 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:13.527908 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0824e-a114-48a3-8803-30e54ea43798 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:13.528189 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0824e-a114-48a3-8803-30e54ea43798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:13.550830 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-36d290e1-2704-488c-9b37-758c2a821aa4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3bfd39123f64e27805ea90036c1340c', 'request_id': 'req-36d290e1-2704-488c-9b37-758c2a821aa4'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:13.574082 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-36d290e1-2704-488c-9b37-758c2a821aa4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3bfd39123f64e27805ea90036c1340c', 'request_id': 'req-36d290e1-2704-488c-9b37-758c2a821aa4'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:13.599035 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-f3f0824e-a114-48a3-8803-30e54ea43798 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:13.616442 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-f3f0824e-a114-48a3-8803-30e54ea43798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:13.723073 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-36d290e1-2704-488c-9b37-758c2a821aa4 admin admin] The token cache is being invalidate because project d3bfd39123f64e27805ea90036c1340c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:27:13.723387 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-36d290e1-2704-488c-9b37-758c2a821aa4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:13.724144 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-36d290e1-2704-488c-9b37-758c2a821aa4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:13.728740 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 850/1679] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:27:13 2022] DELETE /identity/v3/projects/d3bfd39123f64e27805ea90036c1340c => generated 0 bytes in 708 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:27:13.734439 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949d06ec-c2b7-4e36-aa87-26fc054e46c1 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:13.736598 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949d06ec-c2b7-4e36-aa87-26fc054e46c1 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:13.736878 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949d06ec-c2b7-4e36-aa87-26fc054e46c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:13.786155 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-949d06ec-c2b7-4e36-aa87-26fc054e46c1 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:13.788807 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-949d06ec-c2b7-4e36-aa87-26fc054e46c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:13.831915 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 830/1680] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:27:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:27:13.902089 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f0824e-a114-48a3-8803-30e54ea43798 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:13.903689 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-f3f0824e-a114-48a3-8803-30e54ea43798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:13.988915 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 851/1681] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:27:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:27:13.994314 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3040517-d717-4d18-ae84-587cc0b47a10 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:13.997685 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3040517-d717-4d18-ae84-587cc0b47a10 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:13.998192 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3040517-d717-4d18-ae84-587cc0b47a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:14.062038 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-b3040517-d717-4d18-ae84-587cc0b47a10 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:14.063459 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-b3040517-d717-4d18-ae84-587cc0b47a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:14.089900 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 852/1682] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:27:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:27:14.093128 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3040517-d717-4d18-ae84-587cc0b47a10 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:14.116302 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3040517-d717-4d18-ae84-587cc0b47a10 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:14.124126 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f0824e-a114-48a3-8803-30e54ea43798 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:14.133640 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0abd5d-b36c-45e4-9417-3544d60f088c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:14.135445 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0abd5d-b36c-45e4-9417-3544d60f088c admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:14.135818 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0abd5d-b36c-45e4-9417-3544d60f088c admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:14.136664 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0abd5d-b36c-45e4-9417-3544d60f088c admin admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:14.139656 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0abd5d-b36c-45e4-9417-3544d60f088c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:27:14.140037 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0abd5d-b36c-45e4-9417-3544d60f088c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:14.144628 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f768f16-d190-430b-bb51-a2bd8a416ca5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:14.144928 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0abd5d-b36c-45e4-9417-3544d60f088c admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:14.145311 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f768f16-d190-430b-bb51-a2bd8a416ca5 service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:14.145599 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f768f16-d190-430b-bb51-a2bd8a416ca5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:14.145876 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f768f16-d190-430b-bb51-a2bd8a416ca5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:14.166900 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f768f16-d190-430b-bb51-a2bd8a416ca5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:14.168407 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f768f16-d190-430b-bb51-a2bd8a416ca5 service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:14.177487 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 853/1683] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:27:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:27:14.182139 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0abd5d-b36c-45e4-9417-3544d60f088c admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:14.201407 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0abd5d-b36c-45e4-9417-3544d60f088c admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:14.209855 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 831/1684] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:27:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:27:14.215831 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6827d0-af26-482f-82f9-45877eede98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:14.215831 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6827d0-af26-482f-82f9-45877eede98a admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:14.215831 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6827d0-af26-482f-82f9-45877eede98a admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:14.216319 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6827d0-af26-482f-82f9-45877eede98a admin admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:14.217112 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6827d0-af26-482f-82f9-45877eede98a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:14.219647 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6827d0-af26-482f-82f9-45877eede98a admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:14.238227 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:14.325691 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 854/1685] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:27:14 2022] POST /identity/v3/projects => generated 365 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:27:14.334073 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f768f16-d190-430b-bb51-a2bd8a416ca5 service neutron] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:14.342034 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f768f16-d190-430b-bb51-a2bd8a416ca5 service neutron] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:14.356096 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:14.502981 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1260115b-e094-4c67-b9eb-92eca07e9302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:14.505346 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1260115b-e094-4c67-b9eb-92eca07e9302 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:14.505640 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1260115b-e094-4c67-b9eb-92eca07e9302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:14.505924 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1260115b-e094-4c67-b9eb-92eca07e9302 admin admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:14.506720 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1260115b-e094-4c67-b9eb-92eca07e9302 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:14.508914 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1260115b-e094-4c67-b9eb-92eca07e9302 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:14.614194 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a92fa-a318-4ebd-b982-c2026ab03782 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:14.615190 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a92fa-a318-4ebd-b982-c2026ab03782 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:14.615626 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a92fa-a318-4ebd-b982-c2026ab03782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:14.652818 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 832/1686] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:27:14 2022] POST /identity/v3/users => generated 283 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:27:14.655887 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1260115b-e094-4c67-b9eb-92eca07e9302 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:14.671025 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-247a92fa-a318-4ebd-b982-c2026ab03782 None None] MFA Rules not processed for user `35d70344216248199ddf81fba88af04f`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:14.672543 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-247a92fa-a318-4ebd-b982-c2026ab03782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:14.689420 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1260115b-e094-4c67-b9eb-92eca07e9302 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:14.714418 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78d2b44-b2e8-4b54-b0bf-9696b8956e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:14.724063 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d2b44-b2e8-4b54-b0bf-9696b8956e63 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:14.724274 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d2b44-b2e8-4b54-b0bf-9696b8956e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:14.724577 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d2b44-b2e8-4b54-b0bf-9696b8956e63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:14.725276 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78d2b44-b2e8-4b54-b0bf-9696b8956e63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:14.726938 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78d2b44-b2e8-4b54-b0bf-9696b8956e63 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:14.758820 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 833/1687] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:27:14 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:27:14.763077 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78d2b44-b2e8-4b54-b0bf-9696b8956e63 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:14.794034 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78d2b44-b2e8-4b54-b0bf-9696b8956e63 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:14.808023 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05322f0-92eb-4340-8815-d1fb5f920783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:14.809351 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05322f0-92eb-4340-8815-d1fb5f920783 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:14.813124 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05322f0-92eb-4340-8815-d1fb5f920783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:14.813124 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05322f0-92eb-4340-8815-d1fb5f920783 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:14.813124 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05322f0-92eb-4340-8815-d1fb5f920783 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:14.816298 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05322f0-92eb-4340-8815-d1fb5f920783 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:14.865786 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 834/1688] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:27:14 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:27:14.866725 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05322f0-92eb-4340-8815-d1fb5f920783 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:14.869028 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-d05322f0-92eb-4340-8815-d1fb5f920783 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:14.884280 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 855/1689] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:27:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:27:14.885554 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247a92fa-a318-4ebd-b982-c2026ab03782 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:14.889257 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05322f0-92eb-4340-8815-d1fb5f920783 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:14.907557 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358f0821-4fad-483d-a6fe-3a0e61438a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:14.908816 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358f0821-4fad-483d-a6fe-3a0e61438a50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:14.909713 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358f0821-4fad-483d-a6fe-3a0e61438a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:14.911230 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358f0821-4fad-483d-a6fe-3a0e61438a50 admin admin] PATH_INFO: `/v3/projects/ab91aced272948f4b6bd47ffd354162e` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:14.914112 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358f0821-4fad-483d-a6fe-3a0e61438a50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab91aced272948f4b6bd47ffd354162e)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:14.919424 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358f0821-4fad-483d-a6fe-3a0e61438a50 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:14.927030 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247a92fa-a318-4ebd-b982-c2026ab03782 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:14.949099 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f11bac8-8726-41cf-961c-2b70d17b3be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:14.951119 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f11bac8-8726-41cf-961c-2b70d17b3be0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:14.952616 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f11bac8-8726-41cf-961c-2b70d17b3be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:14.953339 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f11bac8-8726-41cf-961c-2b70d17b3be0 admin admin] PATH_INFO: `/v3/projects/fcbc5b26363c49caafbc9af6313470ba/users/1a967b878d0e458dbc182ae62dd46d69/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:14.975920 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f11bac8-8726-41cf-961c-2b70d17b3be0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcbc5b26363c49caafbc9af6313470ba, user_id=1a967b878d0e458dbc182ae62dd46d69, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:14.980918 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f11bac8-8726-41cf-961c-2b70d17b3be0 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:15.177379 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 856/1690] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:27:14 2022] PUT /identity/v3/projects/fcbc5b26363c49caafbc9af6313470ba/users/1a967b878d0e458dbc182ae62dd46d69/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:27:15.178399 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc8121-1289-4d32-89ce-0bf10ab66a97 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:15.178695 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc8121-1289-4d32-89ce-0bf10ab66a97 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:15.178976 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc8121-1289-4d32-89ce-0bf10ab66a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:15.235405 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-358f0821-4fad-483d-a6fe-3a0e61438a50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab91aced272948f4b6bd47ffd354162e', 'request_id': 'req-358f0821-4fad-483d-a6fe-3a0e61438a50'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:15.277066 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-a7fc8121-1289-4d32-89ce-0bf10ab66a97 None None] MFA Rules not processed for user `35d70344216248199ddf81fba88af04f`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:15.278920 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-a7fc8121-1289-4d32-89ce-0bf10ab66a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:15.317348 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-358f0821-4fad-483d-a6fe-3a0e61438a50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab91aced272948f4b6bd47ffd354162e', 'request_id': 'req-358f0821-4fad-483d-a6fe-3a0e61438a50'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:15.331252 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-358f0821-4fad-483d-a6fe-3a0e61438a50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab91aced272948f4b6bd47ffd354162e', 'request_id': 'req-358f0821-4fad-483d-a6fe-3a0e61438a50'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:15.364772 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-358f0821-4fad-483d-a6fe-3a0e61438a50 admin admin] The token cache is being invalidate because project ab91aced272948f4b6bd47ffd354162e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:27:15.365312 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-358f0821-4fad-483d-a6fe-3a0e61438a50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:15.366252 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-358f0821-4fad-483d-a6fe-3a0e61438a50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:15.368313 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 835/1691] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:27:14 2022] DELETE /identity/v3/projects/ab91aced272948f4b6bd47ffd354162e => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:27:15.370484 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0259a4-ecca-44bb-ac6d-28e4e03807aa None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:15.370792 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0259a4-ecca-44bb-ac6d-28e4e03807aa None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:15.371180 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0259a4-ecca-44bb-ac6d-28e4e03807aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:15.430256 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-ce0259a4-ecca-44bb-ac6d-28e4e03807aa None None] MFA Rules not processed for user `1a967b878d0e458dbc182ae62dd46d69`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:15.432267 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-ce0259a4-ecca-44bb-ac6d-28e4e03807aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:15.440905 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 857/1692] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:27:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:27:15.442426 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fc8121-1289-4d32-89ce-0bf10ab66a97 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:15.444110 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-a7fc8121-1289-4d32-89ce-0bf10ab66a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:15.555965 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fc8121-1289-4d32-89ce-0bf10ab66a97 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:15.563504 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2efdbbe-dfba-48c9-b295-ab16dfe67b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:15.564504 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2efdbbe-dfba-48c9-b295-ab16dfe67b89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:15.564786 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2efdbbe-dfba-48c9-b295-ab16dfe67b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:15.565064 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2efdbbe-dfba-48c9-b295-ab16dfe67b89 admin admin] PATH_INFO: `/v3/users/ba16222bd1a948ad887450b8d42dde67` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:15.574161 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2efdbbe-dfba-48c9-b295-ab16dfe67b89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba16222bd1a948ad887450b8d42dde67)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:15.578668 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2efdbbe-dfba-48c9-b295-ab16dfe67b89 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:15.605487 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 836/1693] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:27:15 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:27:15.606905 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0259a4-ecca-44bb-ac6d-28e4e03807aa None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:15.613533 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-ce0259a4-ecca-44bb-ac6d-28e4e03807aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:15.685692 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:15.833312 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0259a4-ecca-44bb-ac6d-28e4e03807aa None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:15.862601 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec9098-3534-484b-ab77-f7d982172dcd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904255dd3c1f406eb10f6fc65f9cd098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:15.863378 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec9098-3534-484b-ab77-f7d982172dcd service nova] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:15.863605 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-e2efdbbe-dfba-48c9-b295-ab16dfe67b89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba16222bd1a948ad887450b8d42dde67'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:15.863855 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec9098-3534-484b-ab77-f7d982172dcd service nova] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:15.864351 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec9098-3534-484b-ab77-f7d982172dcd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:15.868709 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-89ec9098-3534-484b-ab77-f7d982172dcd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:15.898384 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-e2efdbbe-dfba-48c9-b295-ab16dfe67b89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba16222bd1a948ad887450b8d42dde67'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:15.911878 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ec9098-3534-484b-ab77-f7d982172dcd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:15.911878 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ec9098-3534-484b-ab77-f7d982172dcd service nova] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:15.950352 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 837/1694] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:27:15 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:27:15.966075 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec9098-3534-484b-ab77-f7d982172dcd service nova] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:15.970161 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-89ec9098-3534-484b-ab77-f7d982172dcd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:15.985155 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-e2efdbbe-dfba-48c9-b295-ab16dfe67b89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba16222bd1a948ad887450b8d42dde67'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:15.986245 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-e2efdbbe-dfba-48c9-b295-ab16dfe67b89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba16222bd1a948ad887450b8d42dde67'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:16.002600 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-e2efdbbe-dfba-48c9-b295-ab16dfe67b89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba16222bd1a948ad887450b8d42dde67'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:16.019296 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec9098-3534-484b-ab77-f7d982172dcd service nova] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:16.020630 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 858/1695] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:27:15 2022] DELETE /identity/v3/users/ba16222bd1a948ad887450b8d42dde67 => generated 0 bytes in 585 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:27:16.023549 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2efdbbe-dfba-48c9-b295-ab16dfe67b89 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:16.030568 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-e2efdbbe-dfba-48c9-b295-ab16dfe67b89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:16.179075 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:16.221707 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692b7bbf-17c3-4b3f-b0f6-cb2b94d3fd14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:16.223777 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b7bbf-17c3-4b3f-b0f6-cb2b94d3fd14 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:16.225094 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b7bbf-17c3-4b3f-b0f6-cb2b94d3fd14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:16.227235 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b7bbf-17c3-4b3f-b0f6-cb2b94d3fd14 admin admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:16.228796 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692b7bbf-17c3-4b3f-b0f6-cb2b94d3fd14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:16.245672 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2efdbbe-dfba-48c9-b295-ab16dfe67b89 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:16.247324 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692b7bbf-17c3-4b3f-b0f6-cb2b94d3fd14 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:16.260886 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c73be11-d767-48bf-9d6b-5290fea190c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:16.261615 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73be11-d767-48bf-9d6b-5290fea190c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:16.262015 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73be11-d767-48bf-9d6b-5290fea190c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:16.262406 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73be11-d767-48bf-9d6b-5290fea190c7 admin admin] PATH_INFO: `/v3/projects/aa9cddbd0d974e7999c1c0050b0970b4` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:16.264694 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c73be11-d767-48bf-9d6b-5290fea190c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa9cddbd0d974e7999c1c0050b0970b4)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:16.273268 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c73be11-d767-48bf-9d6b-5290fea190c7 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:16.367360 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 838/1696] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:27:15 2022] POST /identity/v3/projects => generated 365 bytes in 416 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:27:16.372358 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30af8a65-9ef8-4cd7-9fa0-5ef8a85ba8c6 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:16.375423 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30af8a65-9ef8-4cd7-9fa0-5ef8a85ba8c6 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:16.375943 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30af8a65-9ef8-4cd7-9fa0-5ef8a85ba8c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:16.409196 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-30af8a65-9ef8-4cd7-9fa0-5ef8a85ba8c6 None None] MFA Rules not processed for user `5c21b9f596da493fac827d3573833ed1`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:16.411505 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-30af8a65-9ef8-4cd7-9fa0-5ef8a85ba8c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:16.498326 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-3c73be11-d767-48bf-9d6b-5290fea190c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa9cddbd0d974e7999c1c0050b0970b4', 'request_id': 'req-3c73be11-d767-48bf-9d6b-5290fea190c7'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:16.514662 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-3c73be11-d767-48bf-9d6b-5290fea190c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa9cddbd0d974e7999c1c0050b0970b4', 'request_id': 'req-3c73be11-d767-48bf-9d6b-5290fea190c7'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:16.529374 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-3c73be11-d767-48bf-9d6b-5290fea190c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa9cddbd0d974e7999c1c0050b0970b4', 'request_id': 'req-3c73be11-d767-48bf-9d6b-5290fea190c7'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:16.537768 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 839/1697] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:27:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:27:16.560764 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30af8a65-9ef8-4cd7-9fa0-5ef8a85ba8c6 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:16.563162 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-30af8a65-9ef8-4cd7-9fa0-5ef8a85ba8c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:16.682379 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-3c73be11-d767-48bf-9d6b-5290fea190c7 admin admin] The token cache is being invalidate because project aa9cddbd0d974e7999c1c0050b0970b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:27:16.682588 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-3c73be11-d767-48bf-9d6b-5290fea190c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:16.686429 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-3c73be11-d767-48bf-9d6b-5290fea190c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:16.692877 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 859/1698] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:27:16 2022] DELETE /identity/v3/projects/aa9cddbd0d974e7999c1c0050b0970b4 => generated 0 bytes in 668 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:27:16.695438 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2bc2c-d513-48bb-8b5e-989a39b2ffc5 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:16.696302 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2bc2c-d513-48bb-8b5e-989a39b2ffc5 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:16.697093 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2bc2c-d513-48bb-8b5e-989a39b2ffc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:16.702003 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30af8a65-9ef8-4cd7-9fa0-5ef8a85ba8c6 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:16.767683 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-cfb2bc2c-d513-48bb-8b5e-989a39b2ffc5 None None] MFA Rules not processed for user `e03c3a73166b4bbe9fca6d18ecf59221`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:16.770072 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-cfb2bc2c-d513-48bb-8b5e-989a39b2ffc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:16.861161 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e5b938-49c3-48c6-92e4-ff512ec50918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:16.862013 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e5b938-49c3-48c6-92e4-ff512ec50918 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:16.862301 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e5b938-49c3-48c6-92e4-ff512ec50918 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:16.862580 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e5b938-49c3-48c6-92e4-ff512ec50918 admin admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:16.863324 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e5b938-49c3-48c6-92e4-ff512ec50918 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:16.868045 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e5b938-49c3-48c6-92e4-ff512ec50918 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:16.898795 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 860/1699] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:27:16 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:27:16.900885 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb2bc2c-d513-48bb-8b5e-989a39b2ffc5 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:16.902448 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-cfb2bc2c-d513-48bb-8b5e-989a39b2ffc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:16.916753 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:17.036377 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 840/1700] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:27:16 2022] POST /identity/v3/users => generated 289 bytes in 498 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:27:17.039514 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca39a7a-d7ef-469f-acee-63a74e64af44 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:17.042257 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca39a7a-d7ef-469f-acee-63a74e64af44 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:17.043071 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca39a7a-d7ef-469f-acee-63a74e64af44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:17.049875 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb2bc2c-d513-48bb-8b5e-989a39b2ffc5 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:17.066622 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b8ca38-9c6e-4ad5-9ea6-795e631e64db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904255dd3c1f406eb10f6fc65f9cd098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:17.067310 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b8ca38-9c6e-4ad5-9ea6-795e631e64db service nova] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:17.067598 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b8ca38-9c6e-4ad5-9ea6-795e631e64db service nova] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:17.067879 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b8ca38-9c6e-4ad5-9ea6-795e631e64db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:17.078839 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-84b8ca38-9c6e-4ad5-9ea6-795e631e64db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:17.086233 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-5ca39a7a-d7ef-469f-acee-63a74e64af44 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:17.089058 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-5ca39a7a-d7ef-469f-acee-63a74e64af44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:17.110663 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 841/1701] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:27:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:27:17.111907 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca39a7a-d7ef-469f-acee-63a74e64af44 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:17.113340 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-5ca39a7a-d7ef-469f-acee-63a74e64af44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:17.133771 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca39a7a-d7ef-469f-acee-63a74e64af44 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:17.140179 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034f42e6-fcfc-4e1e-890b-e033cc51502c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:17.141232 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f42e6-fcfc-4e1e-890b-e033cc51502c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:17.141520 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f42e6-fcfc-4e1e-890b-e033cc51502c admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:17.141802 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f42e6-fcfc-4e1e-890b-e033cc51502c admin admin] PATH_INFO: `/v3/projects/36dbc0ca302a4c0cb5d21a2098dee8f1` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:17.143774 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034f42e6-fcfc-4e1e-890b-e033cc51502c admin admin] RBAC: Authorizing `identity:delete_project(project_id=36dbc0ca302a4c0cb5d21a2098dee8f1)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:17.146057 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034f42e6-fcfc-4e1e-890b-e033cc51502c admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:17.184612 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b8ca38-9c6e-4ad5-9ea6-795e631e64db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:17.186450 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b8ca38-9c6e-4ad5-9ea6-795e631e64db service nova] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:17.197319 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 861/1702] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:27:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:27:17.198330 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b8ca38-9c6e-4ad5-9ea6-795e631e64db service nova] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:17.199711 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-84b8ca38-9c6e-4ad5-9ea6-795e631e64db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:17.231531 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b8ca38-9c6e-4ad5-9ea6-795e631e64db service nova] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:17.244749 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24317767-6a7f-402f-a53d-1d8124592930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:17.247192 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24317767-6a7f-402f-a53d-1d8124592930 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:17.247192 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24317767-6a7f-402f-a53d-1d8124592930 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:17.247192 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24317767-6a7f-402f-a53d-1d8124592930 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:17.247192 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24317767-6a7f-402f-a53d-1d8124592930 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:17.250591 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24317767-6a7f-402f-a53d-1d8124592930 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:17.276270 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 862/1703] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:27:17 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:27:17.280119 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e1726f-3450-47f0-9ff8-f1e51e7f042e None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:17.280441 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e1726f-3450-47f0-9ff8-f1e51e7f042e None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:17.280742 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e1726f-3450-47f0-9ff8-f1e51e7f042e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:17.326616 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-43e1726f-3450-47f0-9ff8-f1e51e7f042e None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:17.330172 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-43e1726f-3450-47f0-9ff8-f1e51e7f042e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:17.348949 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-034f42e6-fcfc-4e1e-890b-e033cc51502c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36dbc0ca302a4c0cb5d21a2098dee8f1', 'request_id': 'req-034f42e6-fcfc-4e1e-890b-e033cc51502c'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:17.454228 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-034f42e6-fcfc-4e1e-890b-e033cc51502c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36dbc0ca302a4c0cb5d21a2098dee8f1', 'request_id': 'req-034f42e6-fcfc-4e1e-890b-e033cc51502c'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:17.461031 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-034f42e6-fcfc-4e1e-890b-e033cc51502c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36dbc0ca302a4c0cb5d21a2098dee8f1', 'request_id': 'req-034f42e6-fcfc-4e1e-890b-e033cc51502c'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:17.473987 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 863/1704] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:27:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:27:17.475912 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa2dca-24f6-4014-99c2-119faf57a812 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:17.477125 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa2dca-24f6-4014-99c2-119faf57a812 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:17.477656 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa2dca-24f6-4014-99c2-119faf57a812 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:17.480891 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 864/1705] 10.222.0.23 () {54 vars in 912 bytes} [Tue Jun 21 18:27:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:27:17.489939 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-034f42e6-fcfc-4e1e-890b-e033cc51502c admin admin] The token cache is being invalidate because project 36dbc0ca302a4c0cb5d21a2098dee8f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:27:17.490234 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-034f42e6-fcfc-4e1e-890b-e033cc51502c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:17.491542 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-034f42e6-fcfc-4e1e-890b-e033cc51502c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:17.492430 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa2dca-24f6-4014-99c2-119faf57a812 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:17.495028 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-40fa2dca-24f6-4014-99c2-119faf57a812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:17.499039 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 842/1706] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:27:17 2022] DELETE /identity/v3/projects/36dbc0ca302a4c0cb5d21a2098dee8f1 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:27:17.507162 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034f42e6-fcfc-4e1e-890b-e033cc51502c admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:17.508067 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-034f42e6-fcfc-4e1e-890b-e033cc51502c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:17.518875 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa2dca-24f6-4014-99c2-119faf57a812 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:17.526100 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf05e221-8bed-4318-bf47-5f2602c578a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:17.527783 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf05e221-8bed-4318-bf47-5f2602c578a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:17.528902 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf05e221-8bed-4318-bf47-5f2602c578a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:17.529301 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf05e221-8bed-4318-bf47-5f2602c578a1 admin admin] PATH_INFO: `/v3/projects/73484ab8023b48f59dce6f650de6f316/users/8361ebcd025e46a399bcd8458e2f93bd/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:17.553431 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf05e221-8bed-4318-bf47-5f2602c578a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73484ab8023b48f59dce6f650de6f316, user_id=8361ebcd025e46a399bcd8458e2f93bd, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:17.557024 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf05e221-8bed-4318-bf47-5f2602c578a1 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:17.656522 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034f42e6-fcfc-4e1e-890b-e033cc51502c admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:17.664135 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be2940c-ff4b-4634-8b90-230ba10d7d9b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904255dd3c1f406eb10f6fc65f9cd098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:17.664963 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be2940c-ff4b-4634-8b90-230ba10d7d9b service nova] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:17.665239 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be2940c-ff4b-4634-8b90-230ba10d7d9b service nova] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:17.666223 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be2940c-ff4b-4634-8b90-230ba10d7d9b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:17.668520 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-4be2940c-ff4b-4634-8b90-230ba10d7d9b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:17.692776 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 865/1707] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:27:17 2022] PUT /identity/v3/projects/73484ab8023b48f59dce6f650de6f316/users/8361ebcd025e46a399bcd8458e2f93bd/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:27:17.697510 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf05e221-8bed-4318-bf47-5f2602c578a1 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:17.710483 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-bf05e221-8bed-4318-bf47-5f2602c578a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:17.785828 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be2940c-ff4b-4634-8b90-230ba10d7d9b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:17.791498 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be2940c-ff4b-4634-8b90-230ba10d7d9b service nova] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:17.809872 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 843/1708] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:27:17 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:27:17.812124 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55268ae6-ba88-42ba-ba69-e32cac552c9b None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:17.812613 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55268ae6-ba88-42ba-ba69-e32cac552c9b None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:17.813039 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55268ae6-ba88-42ba-ba69-e32cac552c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:17.859802 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf05e221-8bed-4318-bf47-5f2602c578a1 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:17.874442 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-55268ae6-ba88-42ba-ba69-e32cac552c9b None None] MFA Rules not processed for user `59c7f07e58ba4db9a7d402d8f3da1e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:17.875933 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-55268ae6-ba88-42ba-ba69-e32cac552c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:17.889156 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1ec4de-6cfd-4dad-8140-96450d0f883b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:17.893334 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ec4de-6cfd-4dad-8140-96450d0f883b admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:17.893988 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ec4de-6cfd-4dad-8140-96450d0f883b admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:17.894320 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ec4de-6cfd-4dad-8140-96450d0f883b admin admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:17.898237 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1ec4de-6cfd-4dad-8140-96450d0f883b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:27:17.900489 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1ec4de-6cfd-4dad-8140-96450d0f883b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:17.906523 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1ec4de-6cfd-4dad-8140-96450d0f883b admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:17.950132 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:17.980365 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 866/1709] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:27:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:27:17.985597 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14df217-42d4-4f95-83c3-2fd9672226ad None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:17.988961 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14df217-42d4-4f95-83c3-2fd9672226ad None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:17.989980 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14df217-42d4-4f95-83c3-2fd9672226ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:18.056891 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-d14df217-42d4-4f95-83c3-2fd9672226ad None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:18.059270 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-d14df217-42d4-4f95-83c3-2fd9672226ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:18.097329 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 844/1710] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Jun 21 18:27:17 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:27:18.099238 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55268ae6-ba88-42ba-ba69-e32cac552c9b None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:18.124024 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 867/1711] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:27:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:27:18.125524 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14df217-42d4-4f95-83c3-2fd9672226ad None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:18.152781 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55268ae6-ba88-42ba-ba69-e32cac552c9b None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:18.157238 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14df217-42d4-4f95-83c3-2fd9672226ad None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:18.163974 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b770878-c9de-49e8-bfa3-040a3194cce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:18.166837 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b770878-c9de-49e8-bfa3-040a3194cce0 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:18.167254 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b770878-c9de-49e8-bfa3-040a3194cce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:18.167646 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b770878-c9de-49e8-bfa3-040a3194cce0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:18.168663 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b770878-c9de-49e8-bfa3-040a3194cce0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:18.171962 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459ff4d6-2b60-44ec-a603-5e27ff003649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:18.172904 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ff4d6-2b60-44ec-a603-5e27ff003649 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:18.173409 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ff4d6-2b60-44ec-a603-5e27ff003649 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:18.173832 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ff4d6-2b60-44ec-a603-5e27ff003649 admin admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:18.174692 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459ff4d6-2b60-44ec-a603-5e27ff003649 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:18.175928 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b770878-c9de-49e8-bfa3-040a3194cce0 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:18.179845 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459ff4d6-2b60-44ec-a603-5e27ff003649 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:18.208360 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 845/1712] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:27:18 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:27:18.209428 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b770878-c9de-49e8-bfa3-040a3194cce0 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:18.217620 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-9b770878-c9de-49e8-bfa3-040a3194cce0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:18.271663 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b770878-c9de-49e8-bfa3-040a3194cce0 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:18.279618 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93074755-159b-4f73-866d-77aa738e2718 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:18.281265 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93074755-159b-4f73-866d-77aa738e2718 service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:18.282132 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93074755-159b-4f73-866d-77aa738e2718 service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:18.283090 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93074755-159b-4f73-866d-77aa738e2718 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:18.289490 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 868/1713] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:27:18 2022] POST /identity/v3/projects => generated 357 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:27:18.291523 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c35668-0cac-41ed-96d1-b16b6562bee8 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:18.291820 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c35668-0cac-41ed-96d1-b16b6562bee8 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:18.292106 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c35668-0cac-41ed-96d1-b16b6562bee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:18.310975 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93074755-159b-4f73-866d-77aa738e2718 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:18.312899 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93074755-159b-4f73-866d-77aa738e2718 service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:18.341430 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 846/1714] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:27:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:27:18.343886 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93074755-159b-4f73-866d-77aa738e2718 service neutron] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:18.350252 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-29c35668-0cac-41ed-96d1-b16b6562bee8 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:18.351487 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-29c35668-0cac-41ed-96d1-b16b6562bee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:18.369295 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93074755-159b-4f73-866d-77aa738e2718 service neutron] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:18.541088 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 869/1715] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:27:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:27:18.542069 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c35668-0cac-41ed-96d1-b16b6562bee8 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:18.553210 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c35668-0cac-41ed-96d1-b16b6562bee8 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:18.558764 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68418a15-2aad-47fc-9453-21da38b9dba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:18.559785 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68418a15-2aad-47fc-9453-21da38b9dba8 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:18.560219 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68418a15-2aad-47fc-9453-21da38b9dba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:18.560810 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68418a15-2aad-47fc-9453-21da38b9dba8 admin admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:18.564639 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68418a15-2aad-47fc-9453-21da38b9dba8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:18.564639 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68418a15-2aad-47fc-9453-21da38b9dba8 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:18.570946 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:18.663241 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca671bd5-919c-4a66-892d-ca7a5ff6da25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:18.666828 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca671bd5-919c-4a66-892d-ca7a5ff6da25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:18.673170 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca671bd5-919c-4a66-892d-ca7a5ff6da25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:18.673170 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca671bd5-919c-4a66-892d-ca7a5ff6da25 admin admin] PATH_INFO: `/v3/projects/73484ab8023b48f59dce6f650de6f316/users/8361ebcd025e46a399bcd8458e2f93bd/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:18.673774 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 870/1716] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:27:18 2022] POST /identity/v3/users => generated 279 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:27:18.673774 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68418a15-2aad-47fc-9453-21da38b9dba8 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:18.680737 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca671bd5-919c-4a66-892d-ca7a5ff6da25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73484ab8023b48f59dce6f650de6f316, user_id=8361ebcd025e46a399bcd8458e2f93bd, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:18.697575 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca671bd5-919c-4a66-892d-ca7a5ff6da25 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:18.699328 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68418a15-2aad-47fc-9453-21da38b9dba8 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:18.716553 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44588697-5034-4d33-8331-7487d0de6af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:18.717553 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44588697-5034-4d33-8331-7487d0de6af9 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:18.717849 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44588697-5034-4d33-8331-7487d0de6af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:18.718134 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44588697-5034-4d33-8331-7487d0de6af9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:18.719070 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44588697-5034-4d33-8331-7487d0de6af9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:27:18.719361 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44588697-5034-4d33-8331-7487d0de6af9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:18.725384 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44588697-5034-4d33-8331-7487d0de6af9 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:18.748668 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 847/1717] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:27:18 2022] PUT /identity/v3/projects/73484ab8023b48f59dce6f650de6f316/users/8361ebcd025e46a399bcd8458e2f93bd/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:27:18.748668 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca671bd5-919c-4a66-892d-ca7a5ff6da25 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:18.772222 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 871/1718] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:27:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:27:18.773787 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b7040-a907-4cad-93d5-0b3460c20053 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:18.774084 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b7040-a907-4cad-93d5-0b3460c20053 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:18.774367 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b7040-a907-4cad-93d5-0b3460c20053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:18.840887 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-039b7040-a907-4cad-93d5-0b3460c20053 None None] MFA Rules not processed for user `8361ebcd025e46a399bcd8458e2f93bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:18.843768 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:18.845845 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-039b7040-a907-4cad-93d5-0b3460c20053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:19.028326 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:19.050673 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 872/1719] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:27:18 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:27:19.050673 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab107c3f-7b36-458a-b03d-c844dc39637b None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:19.050673 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab107c3f-7b36-458a-b03d-c844dc39637b None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:19.054878 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab107c3f-7b36-458a-b03d-c844dc39637b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:19.162585 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca671bd5-919c-4a66-892d-ca7a5ff6da25 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:19.187592 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ffe412-e1e5-43e0-b7cf-de9db0b35536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:19.189868 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ffe412-e1e5-43e0-b7cf-de9db0b35536 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:19.190155 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ffe412-e1e5-43e0-b7cf-de9db0b35536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:19.190433 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ffe412-e1e5-43e0-b7cf-de9db0b35536 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:19.191193 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ffe412-e1e5-43e0-b7cf-de9db0b35536 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:19.193212 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-ab107c3f-7b36-458a-b03d-c844dc39637b None None] MFA Rules not processed for user `59f011e91b684e1288dad5f0ac733c0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:19.200362 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ffe412-e1e5-43e0-b7cf-de9db0b35536 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:19.213879 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-ab107c3f-7b36-458a-b03d-c844dc39637b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:19.239718 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 848/1720] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:27:18 2022] GET /identity/v3/roles => generated 1420 bytes in 505 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:27:19.245149 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ffe412-e1e5-43e0-b7cf-de9db0b35536 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:19.286969 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ffe412-e1e5-43e0-b7cf-de9db0b35536 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:19.329871 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350a7166-da13-44c6-9a41-b1c0d87d7d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:19.334324 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350a7166-da13-44c6-9a41-b1c0d87d7d17 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:19.339300 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350a7166-da13-44c6-9a41-b1c0d87d7d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:19.344331 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350a7166-da13-44c6-9a41-b1c0d87d7d17 admin admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:19.354309 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350a7166-da13-44c6-9a41-b1c0d87d7d17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:19.374177 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350a7166-da13-44c6-9a41-b1c0d87d7d17 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:19.589032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 849/1721] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:27:19 2022] POST /identity/v3/projects => generated 365 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:27:19.590192 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350a7166-da13-44c6-9a41-b1c0d87d7d17 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:19.590744 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 873/1722] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:27:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 546 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:27:19.591601 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab107c3f-7b36-458a-b03d-c844dc39637b None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:19.607953 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-350a7166-da13-44c6-9a41-b1c0d87d7d17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:19.645872 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab107c3f-7b36-458a-b03d-c844dc39637b None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:19.873666 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350a7166-da13-44c6-9a41-b1c0d87d7d17 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:19.876821 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835ad3e9-4c08-496e-9b7f-dc70db1594a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:19.877986 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462bf50f-415c-42f0-97d1-e5d45a2f140b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:19.878667 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462bf50f-415c-42f0-97d1-e5d45a2f140b service cinder] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:19.878956 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462bf50f-415c-42f0-97d1-e5d45a2f140b service cinder] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:19.879237 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462bf50f-415c-42f0-97d1-e5d45a2f140b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:19.880989 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835ad3e9-4c08-496e-9b7f-dc70db1594a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:19.881450 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835ad3e9-4c08-496e-9b7f-dc70db1594a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:19.881884 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835ad3e9-4c08-496e-9b7f-dc70db1594a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:19.882986 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835ad3e9-4c08-496e-9b7f-dc70db1594a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:19.885724 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835ad3e9-4c08-496e-9b7f-dc70db1594a0 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:19.915771 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 874/1723] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:27:19 2022] GET /identity/v3/roles => generated 1420 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:27:19.917043 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835ad3e9-4c08-496e-9b7f-dc70db1594a0 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:19.955652 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835ad3e9-4c08-496e-9b7f-dc70db1594a0 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:19.961824 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b81f140-9662-4da6-b128-9c2e5f83c4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:19.963420 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b81f140-9662-4da6-b128-9c2e5f83c4f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:19.964297 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b81f140-9662-4da6-b128-9c2e5f83c4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:19.964704 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b81f140-9662-4da6-b128-9c2e5f83c4f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:19.965514 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b81f140-9662-4da6-b128-9c2e5f83c4f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:19.967812 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b81f140-9662-4da6-b128-9c2e5f83c4f2 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:20.134798 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462bf50f-415c-42f0-97d1-e5d45a2f140b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:20.136953 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462bf50f-415c-42f0-97d1-e5d45a2f140b service cinder] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:20.142498 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 875/1724] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:27:19 2022] POST /identity/v3/users => generated 283 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:27:20.147702 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a278a3c-431b-4f8e-9203-868976d17773 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:20.150219 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a278a3c-431b-4f8e-9203-868976d17773 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:20.150865 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a278a3c-431b-4f8e-9203-868976d17773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:20.165794 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 850/1725] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:27:19 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 576 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:27:20.182445 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462bf50f-415c-42f0-97d1-e5d45a2f140b service cinder] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:20.198673 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462bf50f-415c-42f0-97d1-e5d45a2f140b service cinder] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:20.207270 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-9a278a3c-431b-4f8e-9203-868976d17773 None None] MFA Rules not processed for user `59f011e91b684e1288dad5f0ac733c0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:20.208961 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a9bdfd-7af3-4dd1-b145-a1e5f073cf00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:20.210024 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9bdfd-7af3-4dd1-b145-a1e5f073cf00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:20.210370 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9bdfd-7af3-4dd1-b145-a1e5f073cf00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:20.210754 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9bdfd-7af3-4dd1-b145-a1e5f073cf00 admin admin] PATH_INFO: `/v3/projects/527104496b274cd5ae43c211cc496d2e/users/1e417704153046589f97cddadcc530d7/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:20.215625 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-9a278a3c-431b-4f8e-9203-868976d17773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:20.230449 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a9bdfd-7af3-4dd1-b145-a1e5f073cf00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=527104496b274cd5ae43c211cc496d2e, user_id=1e417704153046589f97cddadcc530d7, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:20.238225 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a9bdfd-7af3-4dd1-b145-a1e5f073cf00 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:20.365473 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 851/1726] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:27:20 2022] PUT /identity/v3/projects/527104496b274cd5ae43c211cc496d2e/users/1e417704153046589f97cddadcc530d7/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:27:20.368884 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a9bdfd-7af3-4dd1-b145-a1e5f073cf00 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:20.387740 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 876/1727] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:27:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:27:20.391955 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a278a3c-431b-4f8e-9203-868976d17773 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:20.399472 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-9a278a3c-431b-4f8e-9203-868976d17773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:20.557799 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:20.635039 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a9bdfd-7af3-4dd1-b145-a1e5f073cf00 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:20.652439 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a278a3c-431b-4f8e-9203-868976d17773 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:20.663590 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7940948a-0f8e-4957-8eb4-97f1421e9548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:20.664710 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7940948a-0f8e-4957-8eb4-97f1421e9548 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:20.667585 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7940948a-0f8e-4957-8eb4-97f1421e9548 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:20.668363 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7940948a-0f8e-4957-8eb4-97f1421e9548 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:20.669555 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7940948a-0f8e-4957-8eb4-97f1421e9548 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:20.675105 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7940948a-0f8e-4957-8eb4-97f1421e9548 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:20.716026 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 852/1728] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:27:20 2022] GET /identity/v3/roles => generated 1420 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:27:20.719906 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ba94b-edb1-4cdd-a045-e17240926045 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:20.726110 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ba94b-edb1-4cdd-a045-e17240926045 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:20.726601 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ba94b-edb1-4cdd-a045-e17240926045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:20.806816 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-177ba94b-edb1-4cdd-a045-e17240926045 None None] MFA Rules not processed for user `1e417704153046589f97cddadcc530d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:20.809980 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-177ba94b-edb1-4cdd-a045-e17240926045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:20.847169 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b6ee1f-cd31-46e8-ad0b-4b97fdce1a38 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938187967efe49be9c5389d1feef81ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:20.851191 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6ee1f-cd31-46e8-ad0b-4b97fdce1a38 service placement] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:20.854523 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6ee1f-cd31-46e8-ad0b-4b97fdce1a38 service placement] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:20.855323 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6ee1f-cd31-46e8-ad0b-4b97fdce1a38 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:20.864142 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-70b6ee1f-cd31-46e8-ad0b-4b97fdce1a38 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:20.911346 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:21.006776 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b6ee1f-cd31-46e8-ad0b-4b97fdce1a38 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:21.015344 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b6ee1f-cd31-46e8-ad0b-4b97fdce1a38 service placement] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:21.017487 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 853/1729] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:27:20 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:27:21.018377 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ba94b-edb1-4cdd-a045-e17240926045 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:21.037326 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-177ba94b-edb1-4cdd-a045-e17240926045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:21.064594 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 877/1730] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:27:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 676 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:27:21.067667 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b6ee1f-cd31-46e8-ad0b-4b97fdce1a38 service placement] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:21.111493 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b6ee1f-cd31-46e8-ad0b-4b97fdce1a38 service placement] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:21.127259 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1229098-2cfa-4df5-8d24-9e64af4862a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:21.129980 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1229098-2cfa-4df5-8d24-9e64af4862a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:21.130510 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1229098-2cfa-4df5-8d24-9e64af4862a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:21.130994 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1229098-2cfa-4df5-8d24-9e64af4862a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:21.132880 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1229098-2cfa-4df5-8d24-9e64af4862a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:21.138366 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1229098-2cfa-4df5-8d24-9e64af4862a3 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:21.241410 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ba94b-edb1-4cdd-a045-e17240926045 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:21.266131 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 878/1731] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:27:21 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:27:21.273150 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1229098-2cfa-4df5-8d24-9e64af4862a3 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:21.279231 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c04cf4c-8669-4c13-a21f-24ee59dfbb1a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904255dd3c1f406eb10f6fc65f9cd098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:21.279231 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c04cf4c-8669-4c13-a21f-24ee59dfbb1a service nova] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:21.279231 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c04cf4c-8669-4c13-a21f-24ee59dfbb1a service nova] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:21.279231 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c04cf4c-8669-4c13-a21f-24ee59dfbb1a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:21.333830 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1229098-2cfa-4df5-8d24-9e64af4862a3 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:21.354684 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2731a79e-fd70-466a-81cd-da2ddbecda5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:21.355443 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2731a79e-fd70-466a-81cd-da2ddbecda5a admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:21.355732 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2731a79e-fd70-466a-81cd-da2ddbecda5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:21.356012 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2731a79e-fd70-466a-81cd-da2ddbecda5a admin admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:21.356926 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2731a79e-fd70-466a-81cd-da2ddbecda5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:21.358887 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2731a79e-fd70-466a-81cd-da2ddbecda5a admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:21.528971 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c04cf4c-8669-4c13-a21f-24ee59dfbb1a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:21.530716 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c04cf4c-8669-4c13-a21f-24ee59dfbb1a service nova] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:21.538926 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 879/1732] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:27:21 2022] POST /identity/v3/projects => generated 355 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:27:21.540449 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2731a79e-fd70-466a-81cd-da2ddbecda5a admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:21.600323 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 854/1733] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:27:21 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 582 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:27:21.601412 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c04cf4c-8669-4c13-a21f-24ee59dfbb1a service nova] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:21.673986 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c04cf4c-8669-4c13-a21f-24ee59dfbb1a service nova] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:21.791358 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:21.885952 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2731a79e-fd70-466a-81cd-da2ddbecda5a admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:21.920717 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530514bb-2988-4c8b-9896-7014ff2d29f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:21.920717 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530514bb-2988-4c8b-9896-7014ff2d29f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:21.920717 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530514bb-2988-4c8b-9896-7014ff2d29f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:21.920717 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530514bb-2988-4c8b-9896-7014ff2d29f5 admin admin] PATH_INFO: `/v3/projects/a1c1130af97d4a839a8854b481f19339/users/f4bb1382ff67474c824b9a97646663e1/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:21.983228 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530514bb-2988-4c8b-9896-7014ff2d29f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1c1130af97d4a839a8854b481f19339, user_id=f4bb1382ff67474c824b9a97646663e1, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:21.983416 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece4971-cd9e-416e-8400-52770847ebc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:21.983924 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece4971-cd9e-416e-8400-52770847ebc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:21.984211 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece4971-cd9e-416e-8400-52770847ebc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:21.984559 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece4971-cd9e-416e-8400-52770847ebc6 admin admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:21.988139 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ece4971-cd9e-416e-8400-52770847ebc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:21.991939 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530514bb-2988-4c8b-9896-7014ff2d29f5 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:22.009625 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ece4971-cd9e-416e-8400-52770847ebc6 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:22.083049 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:22.154244 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 880/1734] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:27:21 2022] PUT /identity/v3/projects/a1c1130af97d4a839a8854b481f19339/users/f4bb1382ff67474c824b9a97646663e1/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 614 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:27:22.168231 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d229f38a-5890-4adb-8d77-01d01cee8c5a None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:22.168553 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d229f38a-5890-4adb-8d77-01d01cee8c5a None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:22.168881 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d229f38a-5890-4adb-8d77-01d01cee8c5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:22.208102 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 855/1735] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:27:21 2022] POST /identity/v3/users => generated 284 bytes in 607 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:27:22.212596 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece4971-cd9e-416e-8400-52770847ebc6 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:22.254520 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece4971-cd9e-416e-8400-52770847ebc6 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:22.278730 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-d229f38a-5890-4adb-8d77-01d01cee8c5a None None] MFA Rules not processed for user `f4bb1382ff67474c824b9a97646663e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:22.278730 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-d229f38a-5890-4adb-8d77-01d01cee8c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:22.436564 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75170ddd-ce23-40e2-8c7e-7733de0258fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:22.440787 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75170ddd-ce23-40e2-8c7e-7733de0258fc admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:22.441370 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75170ddd-ce23-40e2-8c7e-7733de0258fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:22.441758 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75170ddd-ce23-40e2-8c7e-7733de0258fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:22.442638 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75170ddd-ce23-40e2-8c7e-7733de0258fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:22.450586 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75170ddd-ce23-40e2-8c7e-7733de0258fc admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:22.487379 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 856/1736] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:27:22 2022] GET /identity/v3/roles => generated 1420 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:27:22.488487 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75170ddd-ce23-40e2-8c7e-7733de0258fc admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:22.515641 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 881/1737] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:27:22 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:27:22.518672 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d229f38a-5890-4adb-8d77-01d01cee8c5a None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:22.518799 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75170ddd-ce23-40e2-8c7e-7733de0258fc admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:22.537010 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbda438-0b89-43f4-914a-e13913ac0851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:22.541360 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbda438-0b89-43f4-914a-e13913ac0851 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:22.542016 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbda438-0b89-43f4-914a-e13913ac0851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:22.542384 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d229f38a-5890-4adb-8d77-01d01cee8c5a None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:22.543464 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbda438-0b89-43f4-914a-e13913ac0851 admin admin] PATH_INFO: `/v3/users/1a967b878d0e458dbc182ae62dd46d69` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:22.550726 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbda438-0b89-43f4-914a-e13913ac0851 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a967b878d0e458dbc182ae62dd46d69)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:22.556357 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc8640f-c701-45fc-9aec-7d11a012960d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:22.557758 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc8640f-c701-45fc-9aec-7d11a012960d admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:22.558044 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc8640f-c701-45fc-9aec-7d11a012960d admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:22.558324 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc8640f-c701-45fc-9aec-7d11a012960d admin admin] PATH_INFO: `/v3/projects/45237ca4c8b74688ae0b4643727a5240/users/2c89c6d799ad4189996802de776f7217/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:22.569520 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbda438-0b89-43f4-914a-e13913ac0851 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:22.608872 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc8640f-c701-45fc-9aec-7d11a012960d admin admin] RBAC: Authorizing `identity:create_grant(project_id=45237ca4c8b74688ae0b4643727a5240, user_id=2c89c6d799ad4189996802de776f7217, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:22.614902 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc8640f-c701-45fc-9aec-7d11a012960d admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:22.743172 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 882/1738] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:27:22 2022] PUT /identity/v3/projects/45237ca4c8b74688ae0b4643727a5240/users/2c89c6d799ad4189996802de776f7217/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:27:22.745630 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc8640f-c701-45fc-9aec-7d11a012960d admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:22.849461 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc8640f-c701-45fc-9aec-7d11a012960d admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:22.856824 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609007c0-b9cd-44d4-bcb0-b85c4db931c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:22.857748 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609007c0-b9cd-44d4-bcb0-b85c4db931c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:22.858141 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609007c0-b9cd-44d4-bcb0-b85c4db931c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:22.858511 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609007c0-b9cd-44d4-bcb0-b85c4db931c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:22.859382 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609007c0-b9cd-44d4-bcb0-b85c4db931c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:22.861985 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609007c0-b9cd-44d4-bcb0-b85c4db931c4 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:22.928676 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-ffbda438-0b89-43f4-914a-e13913ac0851 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a967b878d0e458dbc182ae62dd46d69'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:22.976907 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 883/1739] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:27:22 2022] POST /identity/v3/projects => generated 365 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:27:22.982910 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609007c0-b9cd-44d4-bcb0-b85c4db931c4 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:22.997647 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-ffbda438-0b89-43f4-914a-e13913ac0851 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a967b878d0e458dbc182ae62dd46d69'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:23.000986 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-ffbda438-0b89-43f4-914a-e13913ac0851 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a967b878d0e458dbc182ae62dd46d69'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:23.033706 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-ffbda438-0b89-43f4-914a-e13913ac0851 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a967b878d0e458dbc182ae62dd46d69'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:23.034628 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-ffbda438-0b89-43f4-914a-e13913ac0851 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a967b878d0e458dbc182ae62dd46d69'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:23.154107 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 857/1740] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:27:22 2022] DELETE /identity/v3/users/1a967b878d0e458dbc182ae62dd46d69 => generated 0 bytes in 662 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:27:23.156815 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbda438-0b89-43f4-914a-e13913ac0851 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:23.160001 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-ffbda438-0b89-43f4-914a-e13913ac0851 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:23.167634 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609007c0-b9cd-44d4-bcb0-b85c4db931c4 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:23.281101 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbda438-0b89-43f4-914a-e13913ac0851 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:23.286685 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74199617-5fe5-41c6-a396-4bcbc5a07f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:23.287988 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74199617-5fe5-41c6-a396-4bcbc5a07f2c admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:23.291426 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74199617-5fe5-41c6-a396-4bcbc5a07f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:23.292129 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74199617-5fe5-41c6-a396-4bcbc5a07f2c admin admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:23.292962 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74199617-5fe5-41c6-a396-4bcbc5a07f2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:23.295843 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74199617-5fe5-41c6-a396-4bcbc5a07f2c admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:23.314338 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:23.341250 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06046dd-8cef-4081-8159-f474b3d621a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:23.341978 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06046dd-8cef-4081-8159-f474b3d621a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:23.342262 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06046dd-8cef-4081-8159-f474b3d621a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:23.342546 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06046dd-8cef-4081-8159-f474b3d621a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:23.343336 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06046dd-8cef-4081-8159-f474b3d621a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:23.345000 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06046dd-8cef-4081-8159-f474b3d621a9 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:23.361798 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 884/1741] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:27:22 2022] GET /identity/v3/roles => generated 1420 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:27:23.363195 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42b5edd-f8a8-4fbe-8449-5bbf1b5fcdaa None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:23.363524 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42b5edd-f8a8-4fbe-8449-5bbf1b5fcdaa None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:23.363808 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42b5edd-f8a8-4fbe-8449-5bbf1b5fcdaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:23.406341 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-e42b5edd-f8a8-4fbe-8449-5bbf1b5fcdaa None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:23.407884 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-e42b5edd-f8a8-4fbe-8449-5bbf1b5fcdaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:23.425157 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 885/1742] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:27:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:27:23.425382 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 858/1743] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:27:23 2022] POST /identity/v3/users => generated 289 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:27:23.425382 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74199617-5fe5-41c6-a396-4bcbc5a07f2c admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:23.425557 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a627fe4-45f1-4b4f-9794-756e77c696b2 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:23.425557 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a627fe4-45f1-4b4f-9794-756e77c696b2 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:23.428685 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a627fe4-45f1-4b4f-9794-756e77c696b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:23.432136 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-74199617-5fe5-41c6-a396-4bcbc5a07f2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:23.470490 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74199617-5fe5-41c6-a396-4bcbc5a07f2c admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:23.484440 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-5a627fe4-45f1-4b4f-9794-756e77c696b2 None None] MFA Rules not processed for user `5c21b9f596da493fac827d3573833ed1`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:23.485892 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-5a627fe4-45f1-4b4f-9794-756e77c696b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:23.486179 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf01028d-7694-4f20-950d-cc816fff607f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:23.487215 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf01028d-7694-4f20-950d-cc816fff607f admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:23.487748 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf01028d-7694-4f20-950d-cc816fff607f admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:23.488125 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf01028d-7694-4f20-950d-cc816fff607f admin admin] PATH_INFO: `/v3/projects/45237ca4c8b74688ae0b4643727a5240/users/2c89c6d799ad4189996802de776f7217/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:23.498216 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf01028d-7694-4f20-950d-cc816fff607f admin admin] RBAC: Authorizing `identity:create_grant(project_id=45237ca4c8b74688ae0b4643727a5240, user_id=2c89c6d799ad4189996802de776f7217, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:23.502210 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf01028d-7694-4f20-950d-cc816fff607f admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:23.542633 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 859/1744] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:27:23 2022] PUT /identity/v3/projects/45237ca4c8b74688ae0b4643727a5240/users/2c89c6d799ad4189996802de776f7217/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:27:23.543215 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf01028d-7694-4f20-950d-cc816fff607f admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:23.653491 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:23.663491 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 886/1745] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:27:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:27:23.664622 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a627fe4-45f1-4b4f-9794-756e77c696b2 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:23.672811 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-5a627fe4-45f1-4b4f-9794-756e77c696b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:23.710604 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf01028d-7694-4f20-950d-cc816fff607f admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:23.741034 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d209fb2e-ddfa-447d-adf6-7b09758d189d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:23.751184 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d209fb2e-ddfa-447d-adf6-7b09758d189d admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:23.751346 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d209fb2e-ddfa-447d-adf6-7b09758d189d admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:23.751552 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d209fb2e-ddfa-447d-adf6-7b09758d189d admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:23.752406 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d209fb2e-ddfa-447d-adf6-7b09758d189d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:23.771161 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d209fb2e-ddfa-447d-adf6-7b09758d189d admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:23.800968 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 860/1746] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:27:23 2022] GET /identity/v3/roles => generated 1420 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:27:23.807365 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfeb1c8-0a48-4320-b910-c9c8f2b42e3c None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:23.807365 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfeb1c8-0a48-4320-b910-c9c8f2b42e3c None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:23.807365 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfeb1c8-0a48-4320-b910-c9c8f2b42e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:23.911899 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a627fe4-45f1-4b4f-9794-756e77c696b2 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:23.924787 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-1bfeb1c8-0a48-4320-b910-c9c8f2b42e3c None None] MFA Rules not processed for user `2c89c6d799ad4189996802de776f7217`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:23.938235 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-1bfeb1c8-0a48-4320-b910-c9c8f2b42e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:24.071034 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96c95bc-8676-4e88-90e5-a1534ddfe5ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:24.071741 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c95bc-8676-4e88-90e5-a1534ddfe5ac service neutron] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:24.072037 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c95bc-8676-4e88-90e5-a1534ddfe5ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:24.072307 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c95bc-8676-4e88-90e5-a1534ddfe5ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:24.084274 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96c95bc-8676-4e88-90e5-a1534ddfe5ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:24.091559 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96c95bc-8676-4e88-90e5-a1534ddfe5ac service neutron] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:24.105826 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 887/1747] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:27:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:27:24.107736 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e681f2d6-d1d0-4fa9-9232-88c416f48a4a None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:24.108035 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e681f2d6-d1d0-4fa9-9232-88c416f48a4a None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:24.108318 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e681f2d6-d1d0-4fa9-9232-88c416f48a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:24.127971 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 861/1748] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:27:23 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:27:24.127971 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfeb1c8-0a48-4320-b910-c9c8f2b42e3c None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:24.139906 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-e681f2d6-d1d0-4fa9-9232-88c416f48a4a None None] MFA Rules not processed for user `59f011e91b684e1288dad5f0ac733c0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:24.141173 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-e681f2d6-d1d0-4fa9-9232-88c416f48a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:24.157539 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfeb1c8-0a48-4320-b910-c9c8f2b42e3c None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:24.172284 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c3bb43-cb11-400e-a76e-25ac4ba05fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:24.173313 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3bb43-cb11-400e-a76e-25ac4ba05fec admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:24.173521 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3bb43-cb11-400e-a76e-25ac4ba05fec admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:24.173801 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3bb43-cb11-400e-a76e-25ac4ba05fec admin admin] PATH_INFO: `/v3/projects/e9fb99bff2ec42a2ab826c38453f0f5e/users/e7458364cdde41bc8fe68817cb42a20a/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:24.238953 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c3bb43-cb11-400e-a76e-25ac4ba05fec admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9fb99bff2ec42a2ab826c38453f0f5e, user_id=e7458364cdde41bc8fe68817cb42a20a, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:24.251200 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c3bb43-cb11-400e-a76e-25ac4ba05fec admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:24.295420 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 888/1749] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:27:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:27:24.295420 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e681f2d6-d1d0-4fa9-9232-88c416f48a4a None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:24.295420 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-e681f2d6-d1d0-4fa9-9232-88c416f48a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:24.378069 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 862/1750] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:27:24 2022] PUT /identity/v3/projects/e9fb99bff2ec42a2ab826c38453f0f5e/users/e7458364cdde41bc8fe68817cb42a20a/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:27:24.379068 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c3bb43-cb11-400e-a76e-25ac4ba05fec admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:24.404244 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e681f2d6-d1d0-4fa9-9232-88c416f48a4a None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:24.485560 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c3bb43-cb11-400e-a76e-25ac4ba05fec admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:24.505621 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8807cfa7-ba66-4073-ba9f-6ee353a8e126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:24.506515 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8807cfa7-ba66-4073-ba9f-6ee353a8e126 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:24.506810 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8807cfa7-ba66-4073-ba9f-6ee353a8e126 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:24.507090 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8807cfa7-ba66-4073-ba9f-6ee353a8e126 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:24.508670 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8807cfa7-ba66-4073-ba9f-6ee353a8e126 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:24.514920 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8807cfa7-ba66-4073-ba9f-6ee353a8e126 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:24.545793 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:24.547724 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 863/1751] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:27:24 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:27:24.548669 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8807cfa7-ba66-4073-ba9f-6ee353a8e126 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:24.561116 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f6da50-df53-4275-9568-1cde25772bc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:24.561847 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f6da50-df53-4275-9568-1cde25772bc1 service cinder] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:24.562142 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f6da50-df53-4275-9568-1cde25772bc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:24.562424 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f6da50-df53-4275-9568-1cde25772bc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:24.584381 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8807cfa7-ba66-4073-ba9f-6ee353a8e126 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:24.612408 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452b9a2f-5615-4d45-97e2-43a6b38c661a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:24.613781 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b9a2f-5615-4d45-97e2-43a6b38c661a admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:24.614067 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b9a2f-5615-4d45-97e2-43a6b38c661a admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:24.614345 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b9a2f-5615-4d45-97e2-43a6b38c661a admin admin] PATH_INFO: `/v3/projects/e9fb99bff2ec42a2ab826c38453f0f5e/users/e7458364cdde41bc8fe68817cb42a20a/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:24.625024 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452b9a2f-5615-4d45-97e2-43a6b38c661a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9fb99bff2ec42a2ab826c38453f0f5e, user_id=e7458364cdde41bc8fe68817cb42a20a, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:24.637636 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452b9a2f-5615-4d45-97e2-43a6b38c661a admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:24.713320 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 864/1752] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:27:24 2022] PUT /identity/v3/projects/e9fb99bff2ec42a2ab826c38453f0f5e/users/e7458364cdde41bc8fe68817cb42a20a/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:27:24.726205 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04146a69-6d8f-4d38-9a8e-44ce3d67ab56 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:24.726498 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04146a69-6d8f-4d38-9a8e-44ce3d67ab56 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:24.726780 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04146a69-6d8f-4d38-9a8e-44ce3d67ab56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:24.762620 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f6da50-df53-4275-9568-1cde25772bc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:24.764669 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f6da50-df53-4275-9568-1cde25772bc1 service cinder] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:24.801649 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 889/1753] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:27:24 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:27:24.803920 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249ed839-899b-402e-a6ad-71b4bb9404f3 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:24.804376 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249ed839-899b-402e-a6ad-71b4bb9404f3 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:24.804916 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249ed839-899b-402e-a6ad-71b4bb9404f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:24.823833 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-04146a69-6d8f-4d38-9a8e-44ce3d67ab56 None None] MFA Rules not processed for user `e7458364cdde41bc8fe68817cb42a20a`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:24.829962 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-249ed839-899b-402e-a6ad-71b4bb9404f3 None None] MFA Rules not processed for user `e03c3a73166b4bbe9fca6d18ecf59221`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:24.830671 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-04146a69-6d8f-4d38-9a8e-44ce3d67ab56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:24.831656 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-249ed839-899b-402e-a6ad-71b4bb9404f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:24.960592 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 890/1754] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:27:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:27:24.964522 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:25.221946 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249ed839-899b-402e-a6ad-71b4bb9404f3 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:25.225083 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-249ed839-899b-402e-a6ad-71b4bb9404f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:25.245784 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 865/1755] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:27:24 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 521 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:27:25.266067 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04146a69-6d8f-4d38-9a8e-44ce3d67ab56 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:25.393689 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249ed839-899b-402e-a6ad-71b4bb9404f3 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:25.402363 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67691abd-be7d-45f7-b7e7-e5b487ee901b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:25.403187 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67691abd-be7d-45f7-b7e7-e5b487ee901b service cinder] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:25.404998 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67691abd-be7d-45f7-b7e7-e5b487ee901b service cinder] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:25.405878 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67691abd-be7d-45f7-b7e7-e5b487ee901b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:25.413088 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-67691abd-be7d-45f7-b7e7-e5b487ee901b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:25.426211 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04146a69-6d8f-4d38-9a8e-44ce3d67ab56 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:25.435362 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc915b3-3d2f-4ba4-b710-4405193d73ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:25.436518 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc915b3-3d2f-4ba4-b710-4405193d73ab service cinder] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:25.436814 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc915b3-3d2f-4ba4-b710-4405193d73ab service cinder] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:25.437097 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc915b3-3d2f-4ba4-b710-4405193d73ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:25.484658 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc915b3-3d2f-4ba4-b710-4405193d73ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:25.488849 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc915b3-3d2f-4ba4-b710-4405193d73ab service cinder] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:25.528866 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 866/1756] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:27:25 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:27:25.571870 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67691abd-be7d-45f7-b7e7-e5b487ee901b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:25.576018 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67691abd-be7d-45f7-b7e7-e5b487ee901b service cinder] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:25.607433 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 891/1757] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:27:25 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:27:25.750159 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d65ad9e-d38b-473e-a61f-b1fb8122e498 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:25.750770 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d65ad9e-d38b-473e-a61f-b1fb8122e498 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:25.751296 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d65ad9e-d38b-473e-a61f-b1fb8122e498 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:25.753001 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 867/1758] 10.222.0.23 () {54 vars in 912 bytes} [Tue Jun 21 18:27:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:27:25.760902 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b72e6f-9e81-4325-88f4-32937f08decd None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:25.761241 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b72e6f-9e81-4325-88f4-32937f08decd None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:25.761525 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b72e6f-9e81-4325-88f4-32937f08decd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:25.808091 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-01b72e6f-9e81-4325-88f4-32937f08decd None None] MFA Rules not processed for user `59c7f07e58ba4db9a7d402d8f3da1e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:25.810444 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-01b72e6f-9e81-4325-88f4-32937f08decd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:25.864361 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d65ad9e-d38b-473e-a61f-b1fb8122e498 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:25.886178 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d65ad9e-d38b-473e-a61f-b1fb8122e498 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:25.898937 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad4e911-c0ec-43ab-8e25-ca16c735c83a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:25.900147 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4e911-c0ec-43ab-8e25-ca16c735c83a service cinder] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:25.902372 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4e911-c0ec-43ab-8e25-ca16c735c83a service cinder] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:25.910739 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4e911-c0ec-43ab-8e25-ca16c735c83a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:25.913658 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-aad4e911-c0ec-43ab-8e25-ca16c735c83a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:25.954296 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 892/1759] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Jun 21 18:27:25 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:27:25.955107 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b72e6f-9e81-4325-88f4-32937f08decd None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:25.956586 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-01b72e6f-9e81-4325-88f4-32937f08decd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:26.107545 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad4e911-c0ec-43ab-8e25-ca16c735c83a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:26.118935 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad4e911-c0ec-43ab-8e25-ca16c735c83a service cinder] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:26.162884 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 868/1760] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:27:25 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:27:26.170168 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad4e911-c0ec-43ab-8e25-ca16c735c83a service cinder] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:26.189302 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-aad4e911-c0ec-43ab-8e25-ca16c735c83a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:26.214300 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:26.224173 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad4e911-c0ec-43ab-8e25-ca16c735c83a service cinder] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:26.240984 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb03fd3-d033-49b1-8259-0787c0fd9fc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:26.241767 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb03fd3-d033-49b1-8259-0787c0fd9fc0 service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:26.242062 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb03fd3-d033-49b1-8259-0787c0fd9fc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:26.242350 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb03fd3-d033-49b1-8259-0787c0fd9fc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:26.278302 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb03fd3-d033-49b1-8259-0787c0fd9fc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:26.280079 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb03fd3-d033-49b1-8259-0787c0fd9fc0 service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:26.310242 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 869/1761] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:27:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:27:26.375346 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b72e6f-9e81-4325-88f4-32937f08decd None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:26.389679 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b3a524-c52d-49f0-b988-e96b022c9893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:26.393256 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b3a524-c52d-49f0-b988-e96b022c9893 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:26.393256 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b3a524-c52d-49f0-b988-e96b022c9893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:26.393256 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b3a524-c52d-49f0-b988-e96b022c9893 admin admin] PATH_INFO: `/v3/projects/fcbc5b26363c49caafbc9af6313470ba` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:26.398662 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b3a524-c52d-49f0-b988-e96b022c9893 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcbc5b26363c49caafbc9af6313470ba)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:26.404629 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b3a524-c52d-49f0-b988-e96b022c9893 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:26.729798 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-19b3a524-c52d-49f0-b988-e96b022c9893 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcbc5b26363c49caafbc9af6313470ba', 'request_id': 'req-19b3a524-c52d-49f0-b988-e96b022c9893'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:26.745588 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-19b3a524-c52d-49f0-b988-e96b022c9893 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcbc5b26363c49caafbc9af6313470ba', 'request_id': 'req-19b3a524-c52d-49f0-b988-e96b022c9893'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:26.761165 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-19b3a524-c52d-49f0-b988-e96b022c9893 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcbc5b26363c49caafbc9af6313470ba', 'request_id': 'req-19b3a524-c52d-49f0-b988-e96b022c9893'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:26.804484 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:26.907397 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-19b3a524-c52d-49f0-b988-e96b022c9893 admin admin] The token cache is being invalidate because project fcbc5b26363c49caafbc9af6313470ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:27:26.907934 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-19b3a524-c52d-49f0-b988-e96b022c9893 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:26.908769 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-19b3a524-c52d-49f0-b988-e96b022c9893 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:26.911593 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 893/1762] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:27:25 2022] DELETE /identity/v3/projects/fcbc5b26363c49caafbc9af6313470ba => generated 0 bytes in 957 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:27:26.916739 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb03fd3-d033-49b1-8259-0787c0fd9fc0 service neutron] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:26.932978 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb03fd3-d033-49b1-8259-0787c0fd9fc0 service neutron] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:27.025765 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c90f49f-7aa5-4de1-978f-ac6e6bc59d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:27.026982 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90f49f-7aa5-4de1-978f-ac6e6bc59d8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:27.027480 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90f49f-7aa5-4de1-978f-ac6e6bc59d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:27.027905 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90f49f-7aa5-4de1-978f-ac6e6bc59d8d admin admin] PATH_INFO: `/v3/users/8361ebcd025e46a399bcd8458e2f93bd` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:27.029570 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c90f49f-7aa5-4de1-978f-ac6e6bc59d8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8361ebcd025e46a399bcd8458e2f93bd)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:27.031950 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c90f49f-7aa5-4de1-978f-ac6e6bc59d8d admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:27.263501 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-4c90f49f-7aa5-4de1-978f-ac6e6bc59d8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8361ebcd025e46a399bcd8458e2f93bd'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:27.286787 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-4c90f49f-7aa5-4de1-978f-ac6e6bc59d8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8361ebcd025e46a399bcd8458e2f93bd'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:27.288259 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-4c90f49f-7aa5-4de1-978f-ac6e6bc59d8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8361ebcd025e46a399bcd8458e2f93bd'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:27.325840 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-4c90f49f-7aa5-4de1-978f-ac6e6bc59d8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8361ebcd025e46a399bcd8458e2f93bd'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:27.327664 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-4c90f49f-7aa5-4de1-978f-ac6e6bc59d8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8361ebcd025e46a399bcd8458e2f93bd'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:27.444247 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 870/1763] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:27:26 2022] DELETE /identity/v3/users/8361ebcd025e46a399bcd8458e2f93bd => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:27:27.526239 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71acdf8e-753e-4620-9c19-c39b76900d16 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:27.526781 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71acdf8e-753e-4620-9c19-c39b76900d16 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:27.526781 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71acdf8e-753e-4620-9c19-c39b76900d16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:27.557837 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:27.570261 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-71acdf8e-753e-4620-9c19-c39b76900d16 None None] MFA Rules not processed for user `fcfcd9f134e74b09bfdac337c81c8a07`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:27.573005 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-71acdf8e-753e-4620-9c19-c39b76900d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:27.659675 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 894/1764] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Jun 21 18:27:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:27:28.297404 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c90f49f-7aa5-4de1-978f-ac6e6bc59d8d admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:28.298952 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-4c90f49f-7aa5-4de1-978f-ac6e6bc59d8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:28.313608 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:28.470389 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c90f49f-7aa5-4de1-978f-ac6e6bc59d8d admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:28.500310 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc10297-ef76-4d6f-9854-af712678afd5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e0d25565f1543d9a7577288fc09119f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:28.501285 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc10297-ef76-4d6f-9854-af712678afd5 service swift] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:28.501787 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc10297-ef76-4d6f-9854-af712678afd5 service swift] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:28.502258 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc10297-ef76-4d6f-9854-af712678afd5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:28.537242 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc10297-ef76-4d6f-9854-af712678afd5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:28.543276 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc10297-ef76-4d6f-9854-af712678afd5 service swift] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:28.562841 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 871/1765] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Jun 21 18:27:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 18:27:28.638428 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:28.824249 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71acdf8e-753e-4620-9c19-c39b76900d16 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:28.825824 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-71acdf8e-753e-4620-9c19-c39b76900d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:29.098502 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71acdf8e-753e-4620-9c19-c39b76900d16 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:29.121762 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd81e1ca-aa94-4a70-8010-df57e19efb56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:29.122687 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81e1ca-aa94-4a70-8010-df57e19efb56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:29.122977 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81e1ca-aa94-4a70-8010-df57e19efb56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:29.123262 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81e1ca-aa94-4a70-8010-df57e19efb56 admin admin] PATH_INFO: `/v3/projects/73484ab8023b48f59dce6f650de6f316` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:29.133435 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd81e1ca-aa94-4a70-8010-df57e19efb56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73484ab8023b48f59dce6f650de6f316)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:29.135956 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd81e1ca-aa94-4a70-8010-df57e19efb56 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:29.406632 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-fd81e1ca-aa94-4a70-8010-df57e19efb56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73484ab8023b48f59dce6f650de6f316', 'request_id': 'req-fd81e1ca-aa94-4a70-8010-df57e19efb56'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:29.436552 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-fd81e1ca-aa94-4a70-8010-df57e19efb56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73484ab8023b48f59dce6f650de6f316', 'request_id': 'req-fd81e1ca-aa94-4a70-8010-df57e19efb56'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:29.455733 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-fd81e1ca-aa94-4a70-8010-df57e19efb56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73484ab8023b48f59dce6f650de6f316', 'request_id': 'req-fd81e1ca-aa94-4a70-8010-df57e19efb56'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:29.614212 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-fd81e1ca-aa94-4a70-8010-df57e19efb56 admin admin] The token cache is being invalidate because project 73484ab8023b48f59dce6f650de6f316 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:27:29.614401 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-fd81e1ca-aa94-4a70-8010-df57e19efb56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:29.615449 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-fd81e1ca-aa94-4a70-8010-df57e19efb56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:29.636276 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 895/1766] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:27:28 2022] DELETE /identity/v3/projects/73484ab8023b48f59dce6f650de6f316 => generated 0 bytes in 803 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:27:29.637694 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75efbe28-0c99-41e3-8deb-f2cee5c123ee None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:29.637694 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75efbe28-0c99-41e3-8deb-f2cee5c123ee None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:29.637694 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75efbe28-0c99-41e3-8deb-f2cee5c123ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:29.686545 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-75efbe28-0c99-41e3-8deb-f2cee5c123ee None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:29.688184 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-75efbe28-0c99-41e3-8deb-f2cee5c123ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:29.872693 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 872/1767] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:27:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:27:29.883678 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2115e904-ff41-4e18-8393-2f0738c98f29 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:29.884285 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2115e904-ff41-4e18-8393-2f0738c98f29 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:29.884285 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2115e904-ff41-4e18-8393-2f0738c98f29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:29.940442 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-2115e904-ff41-4e18-8393-2f0738c98f29 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:29.941706 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-2115e904-ff41-4e18-8393-2f0738c98f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:29.981831 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 896/1768] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:27:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:27:29.985805 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75efbe28-0c99-41e3-8deb-f2cee5c123ee None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:30.014797 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75efbe28-0c99-41e3-8deb-f2cee5c123ee None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:30.036499 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe7607b-ea25-4a6c-ae9e-e8accba37d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:30.042278 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe7607b-ea25-4a6c-ae9e-e8accba37d53 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:30.042278 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe7607b-ea25-4a6c-ae9e-e8accba37d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:30.042278 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe7607b-ea25-4a6c-ae9e-e8accba37d53 admin admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:30.043403 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe7607b-ea25-4a6c-ae9e-e8accba37d53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:27:30.043867 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe7607b-ea25-4a6c-ae9e-e8accba37d53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:30.046344 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe7607b-ea25-4a6c-ae9e-e8accba37d53 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:30.088675 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2115e904-ff41-4e18-8393-2f0738c98f29 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:30.093267 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 873/1769] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:27:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:27:30.099026 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2115e904-ff41-4e18-8393-2f0738c98f29 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:30.141120 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c83271-0a11-4c72-b64a-4514eb9d5b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:30.146616 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c83271-0a11-4c72-b64a-4514eb9d5b65 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:30.151701 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c83271-0a11-4c72-b64a-4514eb9d5b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:30.152141 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c83271-0a11-4c72-b64a-4514eb9d5b65 admin admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:30.155061 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c83271-0a11-4c72-b64a-4514eb9d5b65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:30.169701 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c83271-0a11-4c72-b64a-4514eb9d5b65 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:30.488451 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 897/1770] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:27:30 2022] POST /identity/v3/projects => generated 385 bytes in 401 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:27:30.519905 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe7607b-ea25-4a6c-ae9e-e8accba37d53 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:30.530902 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe7607b-ea25-4a6c-ae9e-e8accba37d53 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:30.607232 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcd533d-8f67-47bc-aa69-de797ad2d65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:30.608222 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd533d-8f67-47bc-aa69-de797ad2d65f admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:30.609443 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd533d-8f67-47bc-aa69-de797ad2d65f admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:30.610140 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd533d-8f67-47bc-aa69-de797ad2d65f admin admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:30.611069 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcd533d-8f67-47bc-aa69-de797ad2d65f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:30.613718 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcd533d-8f67-47bc-aa69-de797ad2d65f admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:31.100774 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 874/1771] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:27:30 2022] POST /identity/v3/users => generated 293 bytes in 584 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:27:31.104736 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c83271-0a11-4c72-b64a-4514eb9d5b65 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:31.119316 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:31.140418 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c83271-0a11-4c72-b64a-4514eb9d5b65 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:31.159444 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3ad668-7ac1-44f1-9649-99e8c7258ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:31.159444 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ad668-7ac1-44f1-9649-99e8c7258ef1 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:31.159444 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ad668-7ac1-44f1-9649-99e8c7258ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:31.159907 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63db7af8-b530-4e07-98bd-b2ef51fd3288 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:31.159907 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63db7af8-b530-4e07-98bd-b2ef51fd3288 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:31.159907 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63db7af8-b530-4e07-98bd-b2ef51fd3288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:31.170859 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ad668-7ac1-44f1-9649-99e8c7258ef1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:31.180600 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3ad668-7ac1-44f1-9649-99e8c7258ef1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:31.191977 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3ad668-7ac1-44f1-9649-99e8c7258ef1 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:31.222243 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-63db7af8-b530-4e07-98bd-b2ef51fd3288 None None] MFA Rules not processed for user `59c7f07e58ba4db9a7d402d8f3da1e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:31.233180 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-63db7af8-b530-4e07-98bd-b2ef51fd3288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:31.243705 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 898/1772] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:27:31 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:27:31.247321 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3ad668-7ac1-44f1-9649-99e8c7258ef1 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:31.277545 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3ad668-7ac1-44f1-9649-99e8c7258ef1 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:31.291003 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458489ab-eb4e-47a4-bb22-f1a4e3766108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:31.293018 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458489ab-eb4e-47a4-bb22-f1a4e3766108 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:31.295330 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458489ab-eb4e-47a4-bb22-f1a4e3766108 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:31.299447 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458489ab-eb4e-47a4-bb22-f1a4e3766108 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:31.300573 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458489ab-eb4e-47a4-bb22-f1a4e3766108 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:31.305023 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458489ab-eb4e-47a4-bb22-f1a4e3766108 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:31.350314 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 899/1773] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:27:31 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:27:31.354326 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458489ab-eb4e-47a4-bb22-f1a4e3766108 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:31.381629 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:31.396881 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458489ab-eb4e-47a4-bb22-f1a4e3766108 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:31.411979 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 875/1774] 10.222.0.72 () {58 vars in 1015 bytes} [Tue Jun 21 18:27:31 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:27:31.419301 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2bebd5-dbd4-40fc-9024-ac386701cb08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:31.422269 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2bebd5-dbd4-40fc-9024-ac386701cb08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:31.423587 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2bebd5-dbd4-40fc-9024-ac386701cb08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:31.423882 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2bebd5-dbd4-40fc-9024-ac386701cb08 admin admin] PATH_INFO: `/v3/projects/26a8337ba0804a559c73560e09e223d2/users/12a6267e913646fea0bbbad4d8b3ea19/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:31.492125 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2bebd5-dbd4-40fc-9024-ac386701cb08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26a8337ba0804a559c73560e09e223d2, user_id=12a6267e913646fea0bbbad4d8b3ea19, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:31.495332 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2bebd5-dbd4-40fc-9024-ac386701cb08 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:31.651765 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 900/1775] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:27:31 2022] PUT /identity/v3/projects/26a8337ba0804a559c73560e09e223d2/users/12a6267e913646fea0bbbad4d8b3ea19/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:27:31.661401 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beafe1a-bf92-4a0e-addb-3eb126431088 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:31.661950 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beafe1a-bf92-4a0e-addb-3eb126431088 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:31.662381 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beafe1a-bf92-4a0e-addb-3eb126431088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:31.707253 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-2beafe1a-bf92-4a0e-addb-3eb126431088 None None] MFA Rules not processed for user `12a6267e913646fea0bbbad4d8b3ea19`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:31.709019 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-2beafe1a-bf92-4a0e-addb-3eb126431088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:31.829218 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 876/1776] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:27:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:27:31.831513 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2bebd5-dbd4-40fc-9024-ac386701cb08 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:31.871059 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:31.980445 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2bebd5-dbd4-40fc-9024-ac386701cb08 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:31.999185 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242015e8-e8d4-43ac-8572-2cdeef00155e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:32.002441 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242015e8-e8d4-43ac-8572-2cdeef00155e admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:32.002441 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242015e8-e8d4-43ac-8572-2cdeef00155e admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:32.002441 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242015e8-e8d4-43ac-8572-2cdeef00155e admin admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:32.002441 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242015e8-e8d4-43ac-8572-2cdeef00155e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:32.004318 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242015e8-e8d4-43ac-8572-2cdeef00155e admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:32.185971 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 901/1777] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:27:31 2022] POST /identity/v3/projects => generated 385 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:27:32.195322 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beafe1a-bf92-4a0e-addb-3eb126431088 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:32.211696 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beafe1a-bf92-4a0e-addb-3eb126431088 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:32.242041 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242015e8-e8d4-43ac-8572-2cdeef00155e admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:32.245127 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-242015e8-e8d4-43ac-8572-2cdeef00155e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:32.396255 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24c4f37-7dae-466f-abed-a9174755c768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:32.398012 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c4f37-7dae-466f-abed-a9174755c768 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:32.398139 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242015e8-e8d4-43ac-8572-2cdeef00155e admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:32.400768 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c4f37-7dae-466f-abed-a9174755c768 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:32.403921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c4f37-7dae-466f-abed-a9174755c768 admin admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:32.405157 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24c4f37-7dae-466f-abed-a9174755c768 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:32.408628 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24c4f37-7dae-466f-abed-a9174755c768 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:32.425169 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e6a044-f9de-4ec4-b20d-e3e3a7b7d0db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:32.426151 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e6a044-f9de-4ec4-b20d-e3e3a7b7d0db service neutron] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:32.426600 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e6a044-f9de-4ec4-b20d-e3e3a7b7d0db service neutron] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:32.427047 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e6a044-f9de-4ec4-b20d-e3e3a7b7d0db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:32.454295 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e6a044-f9de-4ec4-b20d-e3e3a7b7d0db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:32.455932 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e6a044-f9de-4ec4-b20d-e3e3a7b7d0db service neutron] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:32.486279 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 902/1778] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:27:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:27:32.504450 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 877/1779] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:27:32 2022] POST /identity/v3/users => generated 293 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:27:32.505544 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e6a044-f9de-4ec4-b20d-e3e3a7b7d0db service neutron] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:32.539620 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e6a044-f9de-4ec4-b20d-e3e3a7b7d0db service neutron] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:32.582968 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a617842b-2fe6-40fc-968d-1472740d4b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:32.583839 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a617842b-2fe6-40fc-968d-1472740d4b6c admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:32.584155 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a617842b-2fe6-40fc-968d-1472740d4b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:32.585438 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a617842b-2fe6-40fc-968d-1472740d4b6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:32.585438 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a617842b-2fe6-40fc-968d-1472740d4b6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:32.611310 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a617842b-2fe6-40fc-968d-1472740d4b6c admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:32.663201 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24c4f37-7dae-466f-abed-a9174755c768 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:32.678295 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 903/1780] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:27:32 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:27:32.684365 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24c4f37-7dae-466f-abed-a9174755c768 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:32.698802 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5c5bf5-ec27-4b04-bf3a-a79bf003fcc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:32.699504 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5c5bf5-ec27-4b04-bf3a-a79bf003fcc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:32.699792 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5c5bf5-ec27-4b04-bf3a-a79bf003fcc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:32.700072 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5c5bf5-ec27-4b04-bf3a-a79bf003fcc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:32.705340 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5c5bf5-ec27-4b04-bf3a-a79bf003fcc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:32.715042 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5c5bf5-ec27-4b04-bf3a-a79bf003fcc6 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:32.738888 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a617842b-2fe6-40fc-968d-1472740d4b6c admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:32.747890 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 878/1781] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:27:32 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:27:32.752576 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a617842b-2fe6-40fc-968d-1472740d4b6c admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:32.758840 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8975f627-dab2-4465-af83-974294a6ed2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:32.760180 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8975f627-dab2-4465-af83-974294a6ed2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:32.760768 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8975f627-dab2-4465-af83-974294a6ed2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:32.761939 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8975f627-dab2-4465-af83-974294a6ed2f admin admin] PATH_INFO: `/v3/projects/3b45a0b8f5014fb78e640ba60eea5e58/users/cb39723a51b5460a888a3a8c7ab0e9ef/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:32.854200 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8975f627-dab2-4465-af83-974294a6ed2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b45a0b8f5014fb78e640ba60eea5e58, user_id=cb39723a51b5460a888a3a8c7ab0e9ef, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:32.862126 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8975f627-dab2-4465-af83-974294a6ed2f admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:32.936055 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 904/1782] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:27:32 2022] PUT /identity/v3/projects/3b45a0b8f5014fb78e640ba60eea5e58/users/cb39723a51b5460a888a3a8c7ab0e9ef/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:27:32.947680 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd20510-b369-454b-9507-ed2227cfe231 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:32.948153 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd20510-b369-454b-9507-ed2227cfe231 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:32.948670 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd20510-b369-454b-9507-ed2227cfe231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:33.021504 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-ddd20510-b369-454b-9507-ed2227cfe231 None None] MFA Rules not processed for user `cb39723a51b5460a888a3a8c7ab0e9ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:33.023950 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-ddd20510-b369-454b-9507-ed2227cfe231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:33.188464 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 879/1783] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:27:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:27:33.192592 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8975f627-dab2-4465-af83-974294a6ed2f admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:33.224373 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:33.310812 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8975f627-dab2-4465-af83-974294a6ed2f admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:33.337984 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6941bbe3-7ff1-4af4-877f-7ac44eb17f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:33.337984 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941bbe3-7ff1-4af4-877f-7ac44eb17f0c admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:33.337984 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941bbe3-7ff1-4af4-877f-7ac44eb17f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:33.337984 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941bbe3-7ff1-4af4-877f-7ac44eb17f0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:33.338338 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6941bbe3-7ff1-4af4-877f-7ac44eb17f0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:33.338338 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6941bbe3-7ff1-4af4-877f-7ac44eb17f0c admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:33.416472 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 905/1784] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:27:33 2022] POST /identity/v3/projects => generated 387 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:27:33.419854 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd20510-b369-454b-9507-ed2227cfe231 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:33.431059 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd20510-b369-454b-9507-ed2227cfe231 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:33.518463 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea81f2f-85c8-4cf0-8ed1-732f8d114cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:33.520351 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:33.521065 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea81f2f-85c8-4cf0-8ed1-732f8d114cf2 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:33.521562 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea81f2f-85c8-4cf0-8ed1-732f8d114cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:33.521912 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea81f2f-85c8-4cf0-8ed1-732f8d114cf2 admin admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:33.522701 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea81f2f-85c8-4cf0-8ed1-732f8d114cf2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:33.524886 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea81f2f-85c8-4cf0-8ed1-732f8d114cf2 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:33.647056 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 880/1785] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:27:33 2022] POST /identity/v3/users => generated 300 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:27:33.651962 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6941bbe3-7ff1-4af4-877f-7ac44eb17f0c admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:33.666505 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6941bbe3-7ff1-4af4-877f-7ac44eb17f0c admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:33.678158 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e643f985-dce6-4777-97c8-3264dde57d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:33.679976 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643f985-dce6-4777-97c8-3264dde57d72 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:33.682789 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643f985-dce6-4777-97c8-3264dde57d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:33.683071 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643f985-dce6-4777-97c8-3264dde57d72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:33.685277 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e643f985-dce6-4777-97c8-3264dde57d72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:33.690066 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e643f985-dce6-4777-97c8-3264dde57d72 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:33.711957 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 906/1786] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:27:33.716756 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea81f2f-85c8-4cf0-8ed1-732f8d114cf2 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:33.732874 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea81f2f-85c8-4cf0-8ed1-732f8d114cf2 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:33.742302 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef8d935-b91e-4f60-b0ab-d2978a763665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:33.744190 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef8d935-b91e-4f60-b0ab-d2978a763665 admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:33.744681 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef8d935-b91e-4f60-b0ab-d2978a763665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:33.749651 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef8d935-b91e-4f60-b0ab-d2978a763665 admin admin] PATH_INFO: `/v3/projects/f9be2888eca04455940d6b3775c12062/users/7631cfa60df945b88620db9ae9b47955/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:33.778090 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef8d935-b91e-4f60-b0ab-d2978a763665 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9be2888eca04455940d6b3775c12062, user_id=7631cfa60df945b88620db9ae9b47955, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:33.784272 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef8d935-b91e-4f60-b0ab-d2978a763665 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:33.877409 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 881/1787] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:27:33 2022] PUT /identity/v3/projects/f9be2888eca04455940d6b3775c12062/users/7631cfa60df945b88620db9ae9b47955/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:27:33.882437 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e643f985-dce6-4777-97c8-3264dde57d72 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:33.901949 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e643f985-dce6-4777-97c8-3264dde57d72 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:34.078026 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88582efb-e93f-44fa-bce1-2988b4443327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:34.080152 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88582efb-e93f-44fa-bce1-2988b4443327 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:34.082716 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88582efb-e93f-44fa-bce1-2988b4443327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:34.083237 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88582efb-e93f-44fa-bce1-2988b4443327 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:34.084279 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88582efb-e93f-44fa-bce1-2988b4443327 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:34.088600 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88582efb-e93f-44fa-bce1-2988b4443327 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:34.108531 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef8d935-b91e-4f60-b0ab-d2978a763665 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:34.114449 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 907/1788] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:27:34.121577 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef8d935-b91e-4f60-b0ab-d2978a763665 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:34.130584 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db876fd3-14ae-417c-81e8-9b8e55652a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:34.132909 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db876fd3-14ae-417c-81e8-9b8e55652a2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:34.132909 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db876fd3-14ae-417c-81e8-9b8e55652a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:34.132909 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db876fd3-14ae-417c-81e8-9b8e55652a2f admin admin] PATH_INFO: `/v3/projects/f9be2888eca04455940d6b3775c12062/users/7631cfa60df945b88620db9ae9b47955/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:34.136189 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db876fd3-14ae-417c-81e8-9b8e55652a2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9be2888eca04455940d6b3775c12062, user_id=7631cfa60df945b88620db9ae9b47955, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:34.139501 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db876fd3-14ae-417c-81e8-9b8e55652a2f admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:34.166024 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9574f627-a421-4538-9a0f-26a36a2611e0 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:34.166378 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9574f627-a421-4538-9a0f-26a36a2611e0 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:34.166823 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9574f627-a421-4538-9a0f-26a36a2611e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:34.168804 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 882/1789] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:27:34 2022] PUT /identity/v3/projects/f9be2888eca04455940d6b3775c12062/users/7631cfa60df945b88620db9ae9b47955/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:27:34.224852 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-9574f627-a421-4538-9a0f-26a36a2611e0 None None] MFA Rules not processed for user `7631cfa60df945b88620db9ae9b47955`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:34.226449 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-9574f627-a421-4538-9a0f-26a36a2611e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:34.310640 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:34.385671 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 908/1790] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:27:34 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:27:34.400029 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db876fd3-14ae-417c-81e8-9b8e55652a2f admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:34.406603 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-db876fd3-14ae-417c-81e8-9b8e55652a2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:34.476972 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06da230d-8f1b-4703-a770-e12750bb9ea2 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:34.477862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06da230d-8f1b-4703-a770-e12750bb9ea2 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:34.477862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06da230d-8f1b-4703-a770-e12750bb9ea2 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:34.480247 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 909/1791] 10.222.0.23 () {54 vars in 921 bytes} [Tue Jun 21 18:27:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:27:34.480611 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db876fd3-14ae-417c-81e8-9b8e55652a2f admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:34.484821 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805526c8-807b-4b8f-aa63-7416d2c2d097 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:34.485761 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805526c8-807b-4b8f-aa63-7416d2c2d097 service cinder] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:34.486241 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805526c8-807b-4b8f-aa63-7416d2c2d097 service cinder] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:34.486685 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805526c8-807b-4b8f-aa63-7416d2c2d097 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:34.487174 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd7fa0-debf-4bcc-bf61-e47a4d089d07 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:34.487676 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd7fa0-debf-4bcc-bf61-e47a4d089d07 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:34.488041 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd7fa0-debf-4bcc-bf61-e47a4d089d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:34.499800 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805526c8-807b-4b8f-aa63-7416d2c2d097 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:34.501671 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805526c8-807b-4b8f-aa63-7416d2c2d097 service cinder] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:34.514499 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:34.517692 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 883/1792] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:27:34 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:27:34.530652 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-a1dd7fa0-debf-4bcc-bf61-e47a4d089d07 None None] MFA Rules not processed for user `904255dd3c1f406eb10f6fc65f9cd098`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:34.532503 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-a1dd7fa0-debf-4bcc-bf61-e47a4d089d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:34.708126 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 910/1793] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Jun 21 18:27:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:27:34.720137 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805526c8-807b-4b8f-aa63-7416d2c2d097 service cinder] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:34.724785 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-805526c8-807b-4b8f-aa63-7416d2c2d097 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:34.762350 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805526c8-807b-4b8f-aa63-7416d2c2d097 service cinder] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:34.776954 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eef879-3859-4823-9daa-340945fb0cb8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904255dd3c1f406eb10f6fc65f9cd098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:34.777990 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eef879-3859-4823-9daa-340945fb0cb8 service nova] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:34.778437 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eef879-3859-4823-9daa-340945fb0cb8 service nova] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:34.778888 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eef879-3859-4823-9daa-340945fb0cb8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:34.796008 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eef879-3859-4823-9daa-340945fb0cb8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:34.797840 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eef879-3859-4823-9daa-340945fb0cb8 service nova] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:34.812484 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 884/1794] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:27:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:27:35.959253 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dd7fa0-debf-4bcc-bf61-e47a4d089d07 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:35.960711 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-a1dd7fa0-debf-4bcc-bf61-e47a4d089d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:35.975594 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dd7fa0-debf-4bcc-bf61-e47a4d089d07 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:35.980833 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c263e3d-7c40-4e37-a420-d4efce60970c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:35.981515 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c263e3d-7c40-4e37-a420-d4efce60970c service cinder] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:35.987116 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c263e3d-7c40-4e37-a420-d4efce60970c service cinder] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:35.993324 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c263e3d-7c40-4e37-a420-d4efce60970c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:36.065838 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c263e3d-7c40-4e37-a420-d4efce60970c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:36.068043 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c263e3d-7c40-4e37-a420-d4efce60970c service cinder] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:36.078800 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 911/1795] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:27:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:27:41.239147 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eef879-3859-4823-9daa-340945fb0cb8 service nova] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:41.242339 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-65eef879-3859-4823-9daa-340945fb0cb8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:41.337911 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eef879-3859-4823-9daa-340945fb0cb8 service nova] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:41.349995 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f30b447-b640-4cce-a60d-76b414e9bdef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:41.350834 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f30b447-b640-4cce-a60d-76b414e9bdef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:41.351125 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f30b447-b640-4cce-a60d-76b414e9bdef admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:41.351423 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f30b447-b640-4cce-a60d-76b414e9bdef admin admin] PATH_INFO: `/v3/users/f4bb1382ff67474c824b9a97646663e1` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:41.356759 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f30b447-b640-4cce-a60d-76b414e9bdef admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4bb1382ff67474c824b9a97646663e1)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:41.359073 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f30b447-b640-4cce-a60d-76b414e9bdef admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:41.628230 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-0f30b447-b640-4cce-a60d-76b414e9bdef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4bb1382ff67474c824b9a97646663e1'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:41.642032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:41.667287 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-0f30b447-b640-4cce-a60d-76b414e9bdef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4bb1382ff67474c824b9a97646663e1'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:41.668082 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-0f30b447-b640-4cce-a60d-76b414e9bdef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4bb1382ff67474c824b9a97646663e1'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:41.700586 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-0f30b447-b640-4cce-a60d-76b414e9bdef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4bb1382ff67474c824b9a97646663e1'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:41.702502 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-0f30b447-b640-4cce-a60d-76b414e9bdef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4bb1382ff67474c824b9a97646663e1'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:41.811701 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 885/1796] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:27:41 2022] DELETE /identity/v3/users/f4bb1382ff67474c824b9a97646663e1 => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:27:41.820292 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1777ed5e-27b6-453e-b973-b8812c3f53a2 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:41.820292 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1777ed5e-27b6-453e-b973-b8812c3f53a2 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:41.820292 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1777ed5e-27b6-453e-b973-b8812c3f53a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:41.850778 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-1777ed5e-27b6-453e-b973-b8812c3f53a2 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:41.858108 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-1777ed5e-27b6-453e-b973-b8812c3f53a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:41.942005 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 912/1797] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:27:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:27:42.028861 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f30b447-b640-4cce-a60d-76b414e9bdef admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:42.031133 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-0f30b447-b640-4cce-a60d-76b414e9bdef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:42.098153 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f30b447-b640-4cce-a60d-76b414e9bdef admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:42.102360 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fbd3dc-81f0-465c-a7b2-89f761b6ea80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:42.103246 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fbd3dc-81f0-465c-a7b2-89f761b6ea80 service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:42.103673 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fbd3dc-81f0-465c-a7b2-89f761b6ea80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:42.104006 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fbd3dc-81f0-465c-a7b2-89f761b6ea80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:42.114306 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fbd3dc-81f0-465c-a7b2-89f761b6ea80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:42.115889 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fbd3dc-81f0-465c-a7b2-89f761b6ea80 service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:42.127755 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 886/1798] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:27:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:27:43.657061 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1777ed5e-27b6-453e-b973-b8812c3f53a2 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:43.658641 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-1777ed5e-27b6-453e-b973-b8812c3f53a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:43.747461 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:43.793592 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1777ed5e-27b6-453e-b973-b8812c3f53a2 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:43.817404 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258d4cd1-fccf-49b7-8a01-60838c881690 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:43.818087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258d4cd1-fccf-49b7-8a01-60838c881690 service cinder] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:43.824832 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258d4cd1-fccf-49b7-8a01-60838c881690 service cinder] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:43.825329 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258d4cd1-fccf-49b7-8a01-60838c881690 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:43.837660 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-258d4cd1-fccf-49b7-8a01-60838c881690 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:43.947855 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258d4cd1-fccf-49b7-8a01-60838c881690 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:43.950591 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258d4cd1-fccf-49b7-8a01-60838c881690 service cinder] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:43.963479 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 913/1799] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:27:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:27:44.052888 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fbd3dc-81f0-465c-a7b2-89f761b6ea80 service neutron] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:44.054382 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-93fbd3dc-81f0-465c-a7b2-89f761b6ea80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:44.083976 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fbd3dc-81f0-465c-a7b2-89f761b6ea80 service neutron] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:44.115551 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a221270f-72bf-4f2b-859b-434708d23ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:44.120456 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a221270f-72bf-4f2b-859b-434708d23ef3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:44.120880 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a221270f-72bf-4f2b-859b-434708d23ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:44.121255 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a221270f-72bf-4f2b-859b-434708d23ef3 admin admin] PATH_INFO: `/v3/projects/a1c1130af97d4a839a8854b481f19339` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:44.124156 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a221270f-72bf-4f2b-859b-434708d23ef3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1c1130af97d4a839a8854b481f19339)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:44.126984 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a221270f-72bf-4f2b-859b-434708d23ef3 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:44.250130 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-a221270f-72bf-4f2b-859b-434708d23ef3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1c1130af97d4a839a8854b481f19339', 'request_id': 'req-a221270f-72bf-4f2b-859b-434708d23ef3'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:44.332763 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-a221270f-72bf-4f2b-859b-434708d23ef3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1c1130af97d4a839a8854b481f19339', 'request_id': 'req-a221270f-72bf-4f2b-859b-434708d23ef3'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:44.341379 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-a221270f-72bf-4f2b-859b-434708d23ef3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1c1130af97d4a839a8854b481f19339', 'request_id': 'req-a221270f-72bf-4f2b-859b-434708d23ef3'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:44.361265 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-a221270f-72bf-4f2b-859b-434708d23ef3 admin admin] The token cache is being invalidate because project a1c1130af97d4a839a8854b481f19339 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:27:44.367776 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-a221270f-72bf-4f2b-859b-434708d23ef3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:44.367776 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-a221270f-72bf-4f2b-859b-434708d23ef3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:44.367776 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 887/1800] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:27:44 2022] DELETE /identity/v3/projects/a1c1130af97d4a839a8854b481f19339 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:27:44.369033 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258d4cd1-fccf-49b7-8a01-60838c881690 service cinder] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:44.398229 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258d4cd1-fccf-49b7-8a01-60838c881690 service cinder] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:44.490650 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc83f6b-90e1-4ca0-99d1-c8761e7a79fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:44.491624 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc83f6b-90e1-4ca0-99d1-c8761e7a79fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:44.491905 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc83f6b-90e1-4ca0-99d1-c8761e7a79fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:44.492183 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc83f6b-90e1-4ca0-99d1-c8761e7a79fa admin admin] PATH_INFO: `/v3/users/e7458364cdde41bc8fe68817cb42a20a` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:44.494086 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc83f6b-90e1-4ca0-99d1-c8761e7a79fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7458364cdde41bc8fe68817cb42a20a)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:44.496165 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc83f6b-90e1-4ca0-99d1-c8761e7a79fa admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:44.691459 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-1dc83f6b-90e1-4ca0-99d1-c8761e7a79fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7458364cdde41bc8fe68817cb42a20a'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:44.707023 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-1dc83f6b-90e1-4ca0-99d1-c8761e7a79fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7458364cdde41bc8fe68817cb42a20a'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:44.870959 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-1dc83f6b-90e1-4ca0-99d1-c8761e7a79fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7458364cdde41bc8fe68817cb42a20a'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:44.871673 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-1dc83f6b-90e1-4ca0-99d1-c8761e7a79fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7458364cdde41bc8fe68817cb42a20a'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:44.883889 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-1dc83f6b-90e1-4ca0-99d1-c8761e7a79fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7458364cdde41bc8fe68817cb42a20a'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:44.894714 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 914/1801] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:27:44 2022] DELETE /identity/v3/users/e7458364cdde41bc8fe68817cb42a20a => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:27:45.442601 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a221270f-72bf-4f2b-859b-434708d23ef3 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:45.444755 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-a221270f-72bf-4f2b-859b-434708d23ef3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:45.459319 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a221270f-72bf-4f2b-859b-434708d23ef3 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:45.515125 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:45.521844 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e557521-0cd6-4816-9ce7-241d945a95e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:45.522707 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e557521-0cd6-4816-9ce7-241d945a95e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:45.522988 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e557521-0cd6-4816-9ce7-241d945a95e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:45.523262 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e557521-0cd6-4816-9ce7-241d945a95e3 admin admin] PATH_INFO: `/v3/projects/e9fb99bff2ec42a2ab826c38453f0f5e` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:45.524998 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e557521-0cd6-4816-9ce7-241d945a95e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9fb99bff2ec42a2ab826c38453f0f5e)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:45.526891 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e557521-0cd6-4816-9ce7-241d945a95e3 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:45.710657 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-4e557521-0cd6-4816-9ce7-241d945a95e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9fb99bff2ec42a2ab826c38453f0f5e', 'request_id': 'req-4e557521-0cd6-4816-9ce7-241d945a95e3'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:45.812936 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-4e557521-0cd6-4816-9ce7-241d945a95e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9fb99bff2ec42a2ab826c38453f0f5e', 'request_id': 'req-4e557521-0cd6-4816-9ce7-241d945a95e3'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:45.819610 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-4e557521-0cd6-4816-9ce7-241d945a95e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9fb99bff2ec42a2ab826c38453f0f5e', 'request_id': 'req-4e557521-0cd6-4816-9ce7-241d945a95e3'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:45.840032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-4e557521-0cd6-4816-9ce7-241d945a95e3 admin admin] The token cache is being invalidate because project e9fb99bff2ec42a2ab826c38453f0f5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:27:45.840604 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-4e557521-0cd6-4816-9ce7-241d945a95e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:45.841374 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-4e557521-0cd6-4816-9ce7-241d945a95e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:45.843198 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 888/1802] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:27:45 2022] DELETE /identity/v3/projects/e9fb99bff2ec42a2ab826c38453f0f5e => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:27:45.848569 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f6e84-b018-46bc-9d69-789dcdd9f769 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:45.849113 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f6e84-b018-46bc-9d69-789dcdd9f769 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:45.849113 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f6e84-b018-46bc-9d69-789dcdd9f769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:45.868327 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-249f6e84-b018-46bc-9d69-789dcdd9f769 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:45.869632 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-249f6e84-b018-46bc-9d69-789dcdd9f769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:45.875316 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:45.930154 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 915/1803] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:27:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:27:45.938729 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f3c6f-aa48-486e-8791-cc918a72c8ff None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:45.939021 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f3c6f-aa48-486e-8791-cc918a72c8ff None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:45.939303 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f3c6f-aa48-486e-8791-cc918a72c8ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:45.959181 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-848f3c6f-aa48-486e-8791-cc918a72c8ff None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:45.960491 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-848f3c6f-aa48-486e-8791-cc918a72c8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:45.969071 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 889/1804] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:27:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:27:45.973348 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f6e84-b018-46bc-9d69-789dcdd9f769 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:45.983712 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f6e84-b018-46bc-9d69-789dcdd9f769 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:45.989405 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e4d8fd-f170-4654-a961-b85a535c6a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:45.990070 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e4d8fd-f170-4654-a961-b85a535c6a72 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:45.990365 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e4d8fd-f170-4654-a961-b85a535c6a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:45.990706 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e4d8fd-f170-4654-a961-b85a535c6a72 admin admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:45.991456 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e4d8fd-f170-4654-a961-b85a535c6a72 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:27:45.991764 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e4d8fd-f170-4654-a961-b85a535c6a72 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:45.994139 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e4d8fd-f170-4654-a961-b85a535c6a72 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:46.006119 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 916/1805] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:27:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:27:46.011206 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848f3c6f-aa48-486e-8791-cc918a72c8ff None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:46.020709 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848f3c6f-aa48-486e-8791-cc918a72c8ff None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:46.025676 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7178890f-eb86-4ccc-b9db-64e9affd1fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:46.026351 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7178890f-eb86-4ccc-b9db-64e9affd1fe8 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:46.027958 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7178890f-eb86-4ccc-b9db-64e9affd1fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:46.028243 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7178890f-eb86-4ccc-b9db-64e9affd1fe8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:46.028984 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7178890f-eb86-4ccc-b9db-64e9affd1fe8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:46.030876 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7178890f-eb86-4ccc-b9db-64e9affd1fe8 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:46.431726 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 890/1806] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:27:46 2022] POST /identity/v3/projects => generated 377 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:27:46.443683 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e4d8fd-f170-4654-a961-b85a535c6a72 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:46.455528 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e4d8fd-f170-4654-a961-b85a535c6a72 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:46.536189 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdc2fe2-eeed-4f3d-ba10-2378b97d7d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:46.536848 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc2fe2-eeed-4f3d-ba10-2378b97d7d98 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:46.537098 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc2fe2-eeed-4f3d-ba10-2378b97d7d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:46.537463 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc2fe2-eeed-4f3d-ba10-2378b97d7d98 admin admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:46.539055 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdc2fe2-eeed-4f3d-ba10-2378b97d7d98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:46.541640 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdc2fe2-eeed-4f3d-ba10-2378b97d7d98 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:46.693563 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 917/1807] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:27:46 2022] POST /identity/v3/users => generated 289 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:27:46.697800 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7178890f-eb86-4ccc-b9db-64e9affd1fe8 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:46.708455 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7178890f-eb86-4ccc-b9db-64e9affd1fe8 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:46.714242 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1caf0d1-d60a-4c23-a29b-713216fd8576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:46.714549 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1caf0d1-d60a-4c23-a29b-713216fd8576 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:46.714762 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1caf0d1-d60a-4c23-a29b-713216fd8576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:46.715166 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1caf0d1-d60a-4c23-a29b-713216fd8576 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:46.716207 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1caf0d1-d60a-4c23-a29b-713216fd8576 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:46.717845 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1caf0d1-d60a-4c23-a29b-713216fd8576 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:46.729452 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 891/1808] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:27:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:27:46.733674 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdc2fe2-eeed-4f3d-ba10-2378b97d7d98 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:46.744642 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdc2fe2-eeed-4f3d-ba10-2378b97d7d98 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:46.749712 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551256d5-b271-46d4-9f4a-7eaa1436b8ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:46.754551 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551256d5-b271-46d4-9f4a-7eaa1436b8ea admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:46.754551 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551256d5-b271-46d4-9f4a-7eaa1436b8ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:46.754551 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551256d5-b271-46d4-9f4a-7eaa1436b8ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:46.754551 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551256d5-b271-46d4-9f4a-7eaa1436b8ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:46.754551 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551256d5-b271-46d4-9f4a-7eaa1436b8ea admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:46.764068 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 918/1809] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:27:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:27:46.768262 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1caf0d1-d60a-4c23-a29b-713216fd8576 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:46.779169 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1caf0d1-d60a-4c23-a29b-713216fd8576 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:46.784271 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dd940b-f668-42ce-b0a1-bfdb78398c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:46.785196 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd940b-f668-42ce-b0a1-bfdb78398c36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:46.785552 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd940b-f668-42ce-b0a1-bfdb78398c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:46.785835 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd940b-f668-42ce-b0a1-bfdb78398c36 admin admin] PATH_INFO: `/v3/projects/8378f00d14b844a5ac3d20add3fc208a/users/646952e2ed8b4f99a7bec0bd4abbda95/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:46.801870 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dd940b-f668-42ce-b0a1-bfdb78398c36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8378f00d14b844a5ac3d20add3fc208a, user_id=646952e2ed8b4f99a7bec0bd4abbda95, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:46.804735 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dd940b-f668-42ce-b0a1-bfdb78398c36 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:46.894910 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 892/1810] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:27:46 2022] PUT /identity/v3/projects/8378f00d14b844a5ac3d20add3fc208a/users/646952e2ed8b4f99a7bec0bd4abbda95/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:27:46.905842 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba759a-901e-438d-a135-2e45c00abfd6 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:46.906466 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba759a-901e-438d-a135-2e45c00abfd6 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:46.906466 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba759a-901e-438d-a135-2e45c00abfd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:46.935211 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-3aba759a-901e-438d-a135-2e45c00abfd6 None None] MFA Rules not processed for user `646952e2ed8b4f99a7bec0bd4abbda95`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:46.936416 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-3aba759a-901e-438d-a135-2e45c00abfd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:47.001919 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 919/1811] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:27:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:27:47.006401 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dd940b-f668-42ce-b0a1-bfdb78398c36 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:47.070946 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dd940b-f668-42ce-b0a1-bfdb78398c36 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:47.076073 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e524981-59d7-4706-978e-5d59b6b305cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:47.076779 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e524981-59d7-4706-978e-5d59b6b305cd admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:47.077071 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e524981-59d7-4706-978e-5d59b6b305cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:47.077355 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e524981-59d7-4706-978e-5d59b6b305cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:47.078049 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e524981-59d7-4706-978e-5d59b6b305cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:47.080071 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e524981-59d7-4706-978e-5d59b6b305cd admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:47.162706 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 893/1812] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:27:47 2022] POST /identity/v3/projects => generated 377 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:27:47.167253 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aba759a-901e-438d-a135-2e45c00abfd6 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:47.177938 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aba759a-901e-438d-a135-2e45c00abfd6 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:47.182865 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:47.236352 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9b1b3e-1410-4ad0-b841-293060ffdbb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:47.236978 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b1b3e-1410-4ad0-b841-293060ffdbb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:47.237268 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b1b3e-1410-4ad0-b841-293060ffdbb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:47.237551 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b1b3e-1410-4ad0-b841-293060ffdbb6 admin admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:47.238264 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9b1b3e-1410-4ad0-b841-293060ffdbb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:47.240170 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9b1b3e-1410-4ad0-b841-293060ffdbb6 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:47.327388 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 920/1813] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:27:47 2022] POST /identity/v3/users => generated 289 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:27:47.331074 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e524981-59d7-4706-978e-5d59b6b305cd admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:47.342332 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e524981-59d7-4706-978e-5d59b6b305cd admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:47.346988 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6361edb4-115f-4aea-83d9-0a43cc9986da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:47.347650 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6361edb4-115f-4aea-83d9-0a43cc9986da admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:47.347935 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6361edb4-115f-4aea-83d9-0a43cc9986da admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:47.348317 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6361edb4-115f-4aea-83d9-0a43cc9986da admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:47.349073 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6361edb4-115f-4aea-83d9-0a43cc9986da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:47.350686 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6361edb4-115f-4aea-83d9-0a43cc9986da admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:47.351924 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:47.362501 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 894/1814] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:27:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:27:47.366858 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9b1b3e-1410-4ad0-b841-293060ffdbb6 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:47.377520 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9b1b3e-1410-4ad0-b841-293060ffdbb6 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:47.382748 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607187bb-214b-498b-a3c0-a8944dc05f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:47.383929 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607187bb-214b-498b-a3c0-a8944dc05f9d admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:47.384855 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607187bb-214b-498b-a3c0-a8944dc05f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:47.385134 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607187bb-214b-498b-a3c0-a8944dc05f9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:47.385864 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607187bb-214b-498b-a3c0-a8944dc05f9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:47.387456 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607187bb-214b-498b-a3c0-a8944dc05f9d admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:47.397402 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 921/1815] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:27:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:27:47.402475 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6361edb4-115f-4aea-83d9-0a43cc9986da admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:47.412296 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6361edb4-115f-4aea-83d9-0a43cc9986da admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:47.417867 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9012776a-8aa9-40f0-9d2d-834375e5ea95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:47.418752 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9012776a-8aa9-40f0-9d2d-834375e5ea95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:47.427496 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9012776a-8aa9-40f0-9d2d-834375e5ea95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:47.427496 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9012776a-8aa9-40f0-9d2d-834375e5ea95 admin admin] PATH_INFO: `/v3/projects/89a007f9ce194dadb2dba2850c38d11d/users/78eb338bf15849b3bd58e33927c289bc/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:47.433703 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9012776a-8aa9-40f0-9d2d-834375e5ea95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89a007f9ce194dadb2dba2850c38d11d, user_id=78eb338bf15849b3bd58e33927c289bc, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:47.436347 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9012776a-8aa9-40f0-9d2d-834375e5ea95 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:47.509733 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 895/1816] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:27:47 2022] PUT /identity/v3/projects/89a007f9ce194dadb2dba2850c38d11d/users/78eb338bf15849b3bd58e33927c289bc/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:27:47.520354 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841c739d-d702-4889-af5f-75166e8f3c41 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:47.520639 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841c739d-d702-4889-af5f-75166e8f3c41 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:47.520914 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841c739d-d702-4889-af5f-75166e8f3c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:47.554033 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-841c739d-d702-4889-af5f-75166e8f3c41 None None] MFA Rules not processed for user `78eb338bf15849b3bd58e33927c289bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:47.555421 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-841c739d-d702-4889-af5f-75166e8f3c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:47.664155 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 922/1817] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:27:47 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:27:47.665153 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9012776a-8aa9-40f0-9d2d-834375e5ea95 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:47.757733 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9012776a-8aa9-40f0-9d2d-834375e5ea95 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:47.766680 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b255ab7-50be-44d6-8b09-dc78798debba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:47.767679 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b255ab7-50be-44d6-8b09-dc78798debba admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:47.768122 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b255ab7-50be-44d6-8b09-dc78798debba admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:47.768604 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b255ab7-50be-44d6-8b09-dc78798debba admin admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:47.769587 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b255ab7-50be-44d6-8b09-dc78798debba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:47.772106 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b255ab7-50be-44d6-8b09-dc78798debba admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:47.877947 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 896/1818] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:27:47 2022] POST /identity/v3/projects => generated 377 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:27:47.879596 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841c739d-d702-4889-af5f-75166e8f3c41 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:47.896890 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841c739d-d702-4889-af5f-75166e8f3c41 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:47.992060 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa469c3-8a67-40ba-ae7b-8495ce79d1bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:47.997871 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa469c3-8a67-40ba-ae7b-8495ce79d1bb admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:47.998190 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa469c3-8a67-40ba-ae7b-8495ce79d1bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:47.998475 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa469c3-8a67-40ba-ae7b-8495ce79d1bb admin admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:47.999600 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa469c3-8a67-40ba-ae7b-8495ce79d1bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:48.009114 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa469c3-8a67-40ba-ae7b-8495ce79d1bb admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:48.099330 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 923/1819] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:27:47 2022] POST /identity/v3/users => generated 295 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:27:48.100660 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b255ab7-50be-44d6-8b09-dc78798debba admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:48.112724 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b255ab7-50be-44d6-8b09-dc78798debba admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:48.119730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceaa0f8-4e69-430f-bed9-a8b8032752ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:48.120461 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceaa0f8-4e69-430f-bed9-a8b8032752ed admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:48.120756 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceaa0f8-4e69-430f-bed9-a8b8032752ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:48.121034 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceaa0f8-4e69-430f-bed9-a8b8032752ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:48.121781 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceaa0f8-4e69-430f-bed9-a8b8032752ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:48.123459 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceaa0f8-4e69-430f-bed9-a8b8032752ed admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:48.138320 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 897/1820] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:27:48.142236 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa469c3-8a67-40ba-ae7b-8495ce79d1bb admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:48.149692 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:48.155710 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa469c3-8a67-40ba-ae7b-8495ce79d1bb admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:48.166661 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccef7f5a-4708-42d8-b938-78763ee7cf22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:48.168091 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef7f5a-4708-42d8-b938-78763ee7cf22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:48.168789 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef7f5a-4708-42d8-b938-78763ee7cf22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:48.169077 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef7f5a-4708-42d8-b938-78763ee7cf22 admin admin] PATH_INFO: `/v3/projects/456420aac8ec4e02a08b07824f235417/users/7477c79e28c648ff85d1178d5177e832/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:48.189671 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccef7f5a-4708-42d8-b938-78763ee7cf22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=456420aac8ec4e02a08b07824f235417, user_id=7477c79e28c648ff85d1178d5177e832, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:48.192634 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccef7f5a-4708-42d8-b938-78763ee7cf22 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:48.278722 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 924/1821] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:27:48 2022] PUT /identity/v3/projects/456420aac8ec4e02a08b07824f235417/users/7477c79e28c648ff85d1178d5177e832/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:27:48.282768 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceaa0f8-4e69-430f-bed9-a8b8032752ed admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:48.294060 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceaa0f8-4e69-430f-bed9-a8b8032752ed admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:48.386374 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52932f57-c087-4f84-a924-85d59eb92158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:48.387710 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52932f57-c087-4f84-a924-85d59eb92158 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:48.388081 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52932f57-c087-4f84-a924-85d59eb92158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:48.398598 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52932f57-c087-4f84-a924-85d59eb92158 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:48.398598 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52932f57-c087-4f84-a924-85d59eb92158 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:48.398598 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52932f57-c087-4f84-a924-85d59eb92158 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:48.416176 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 898/1822] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:27:48.420734 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccef7f5a-4708-42d8-b938-78763ee7cf22 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:48.440116 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccef7f5a-4708-42d8-b938-78763ee7cf22 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:48.463590 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f873b314-5c83-4833-a1cc-5f1b023e0372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:48.466577 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f873b314-5c83-4833-a1cc-5f1b023e0372 admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:48.466865 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f873b314-5c83-4833-a1cc-5f1b023e0372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:48.467147 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f873b314-5c83-4833-a1cc-5f1b023e0372 admin admin] PATH_INFO: `/v3/projects/456420aac8ec4e02a08b07824f235417/users/7477c79e28c648ff85d1178d5177e832/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:48.475581 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f873b314-5c83-4833-a1cc-5f1b023e0372 admin admin] RBAC: Authorizing `identity:create_grant(project_id=456420aac8ec4e02a08b07824f235417, user_id=7477c79e28c648ff85d1178d5177e832, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:48.480847 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f873b314-5c83-4833-a1cc-5f1b023e0372 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:48.497659 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 925/1823] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:27:48 2022] PUT /identity/v3/projects/456420aac8ec4e02a08b07824f235417/users/7477c79e28c648ff85d1178d5177e832/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:27:48.507950 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b87c8-07bb-4c54-9274-e5a8e6325fc9 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:48.508422 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b87c8-07bb-4c54-9274-e5a8e6325fc9 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:48.508719 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b87c8-07bb-4c54-9274-e5a8e6325fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:48.554072 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-785b87c8-07bb-4c54-9274-e5a8e6325fc9 None None] MFA Rules not processed for user `7477c79e28c648ff85d1178d5177e832`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:48.555706 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-785b87c8-07bb-4c54-9274-e5a8e6325fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:48.599259 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:48.653831 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 899/1824] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:27:48 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:27:48.666036 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f873b314-5c83-4833-a1cc-5f1b023e0372 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:48.669418 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-f873b314-5c83-4833-a1cc-5f1b023e0372 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:48.786239 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f873b314-5c83-4833-a1cc-5f1b023e0372 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:48.790991 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b05ca-d189-4759-98f5-2259c8dc6dd3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:48.795846 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b05ca-d189-4759-98f5-2259c8dc6dd3 service cinder] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:48.795846 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b05ca-d189-4759-98f5-2259c8dc6dd3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:48.795846 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b05ca-d189-4759-98f5-2259c8dc6dd3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:48.813956 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9b05ca-d189-4759-98f5-2259c8dc6dd3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:48.825581 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9b05ca-d189-4759-98f5-2259c8dc6dd3 service cinder] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:48.839315 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 926/1825] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:27:48 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:27:48.959997 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1a95ae-7f76-490f-b5a9-550aa6398117 None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:48.961208 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1a95ae-7f76-490f-b5a9-550aa6398117 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:48.962254 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1a95ae-7f76-490f-b5a9-550aa6398117 None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:48.964868 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 900/1826] 10.222.0.23 () {54 vars in 921 bytes} [Tue Jun 21 18:27:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:27:48.988187 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb60963-507c-4af0-a6fb-e44bfe638818 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:48.988745 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb60963-507c-4af0-a6fb-e44bfe638818 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:48.989166 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb60963-507c-4af0-a6fb-e44bfe638818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:49.000017 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1a95ae-7f76-490f-b5a9-550aa6398117 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:49.028522 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1a95ae-7f76-490f-b5a9-550aa6398117 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:49.034747 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd13fd9d-9088-4c94-90dc-9680947b0b6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:49.040055 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13fd9d-9088-4c94-90dc-9680947b0b6a service cinder] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:49.040055 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13fd9d-9088-4c94-90dc-9680947b0b6a service cinder] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:49.040055 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13fd9d-9088-4c94-90dc-9680947b0b6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:49.041071 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-fbb60963-507c-4af0-a6fb-e44bfe638818 None None] MFA Rules not processed for user `904255dd3c1f406eb10f6fc65f9cd098`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:49.041646 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-fbb60963-507c-4af0-a6fb-e44bfe638818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:49.185209 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd13fd9d-9088-4c94-90dc-9680947b0b6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:49.186937 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd13fd9d-9088-4c94-90dc-9680947b0b6a service cinder] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:49.206349 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 901/1827] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:27:48 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:27:49.206472 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 927/1828] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Jun 21 18:27:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:27:49.235932 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd13fd9d-9088-4c94-90dc-9680947b0b6a service cinder] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:49.237630 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-bd13fd9d-9088-4c94-90dc-9680947b0b6a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:49.265370 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd13fd9d-9088-4c94-90dc-9680947b0b6a service cinder] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:49.266151 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ae9455-afbd-4857-bb0b-c7b90408b803 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904255dd3c1f406eb10f6fc65f9cd098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:49.266957 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae9455-afbd-4857-bb0b-c7b90408b803 service nova] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:49.267521 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae9455-afbd-4857-bb0b-c7b90408b803 service nova] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:49.267807 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae9455-afbd-4857-bb0b-c7b90408b803 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:49.280820 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ae9455-afbd-4857-bb0b-c7b90408b803 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:49.282874 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ae9455-afbd-4857-bb0b-c7b90408b803 service nova] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:49.298147 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 902/1829] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:27:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:27:49.722684 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb60963-507c-4af0-a6fb-e44bfe638818 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:49.725897 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-fbb60963-507c-4af0-a6fb-e44bfe638818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:49.793480 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb60963-507c-4af0-a6fb-e44bfe638818 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:49.798015 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a205dd-55fb-41c1-9b85-910f650af177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:49.798889 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a205dd-55fb-41c1-9b85-910f650af177 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:49.799172 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a205dd-55fb-41c1-9b85-910f650af177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:49.799446 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a205dd-55fb-41c1-9b85-910f650af177 admin admin] PATH_INFO: `/v3/users/12a6267e913646fea0bbbad4d8b3ea19` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:49.801177 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a205dd-55fb-41c1-9b85-910f650af177 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12a6267e913646fea0bbbad4d8b3ea19)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:49.802594 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a205dd-55fb-41c1-9b85-910f650af177 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:49.805043 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:49.983492 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-c2a205dd-55fb-41c1-9b85-910f650af177 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12a6267e913646fea0bbbad4d8b3ea19'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:49.998216 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-c2a205dd-55fb-41c1-9b85-910f650af177 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12a6267e913646fea0bbbad4d8b3ea19'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:50.046925 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-c2a205dd-55fb-41c1-9b85-910f650af177 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12a6267e913646fea0bbbad4d8b3ea19'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:50.048530 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-c2a205dd-55fb-41c1-9b85-910f650af177 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12a6267e913646fea0bbbad4d8b3ea19'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:50.058133 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-c2a205dd-55fb-41c1-9b85-910f650af177 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12a6267e913646fea0bbbad4d8b3ea19'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:50.065106 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 928/1830] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:27:49 2022] DELETE /identity/v3/users/12a6267e913646fea0bbbad4d8b3ea19 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:27:50.069892 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f866e-a3ae-4c36-9cf6-d30cdd3355b2 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:50.072074 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f866e-a3ae-4c36-9cf6-d30cdd3355b2 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:50.072074 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f866e-a3ae-4c36-9cf6-d30cdd3355b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:50.091364 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-c44f866e-a3ae-4c36-9cf6-d30cdd3355b2 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:50.092738 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-c44f866e-a3ae-4c36-9cf6-d30cdd3355b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:50.172267 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 903/1831] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:27:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:27:50.182932 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a205dd-55fb-41c1-9b85-910f650af177 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:50.184544 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-c2a205dd-55fb-41c1-9b85-910f650af177 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:50.252072 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a205dd-55fb-41c1-9b85-910f650af177 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:50.255741 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97cedf3-bfde-4e14-89d6-d5c39de65a60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:50.256419 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97cedf3-bfde-4e14-89d6-d5c39de65a60 service neutron] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:50.256717 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97cedf3-bfde-4e14-89d6-d5c39de65a60 service neutron] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:50.256992 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97cedf3-bfde-4e14-89d6-d5c39de65a60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:50.266273 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97cedf3-bfde-4e14-89d6-d5c39de65a60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:50.267692 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97cedf3-bfde-4e14-89d6-d5c39de65a60 service neutron] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:50.277965 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 929/1832] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:27:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:27:50.842650 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44f866e-a3ae-4c36-9cf6-d30cdd3355b2 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:50.844796 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-c44f866e-a3ae-4c36-9cf6-d30cdd3355b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:50.873085 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44f866e-a3ae-4c36-9cf6-d30cdd3355b2 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:50.879467 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41fbf4c-74e7-4afa-8ede-d6c7bebaec6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:50.881497 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41fbf4c-74e7-4afa-8ede-d6c7bebaec6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:50.887056 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41fbf4c-74e7-4afa-8ede-d6c7bebaec6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:50.887056 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41fbf4c-74e7-4afa-8ede-d6c7bebaec6c admin admin] PATH_INFO: `/v3/projects/26a8337ba0804a559c73560e09e223d2` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:50.887056 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41fbf4c-74e7-4afa-8ede-d6c7bebaec6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=26a8337ba0804a559c73560e09e223d2)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:50.888762 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41fbf4c-74e7-4afa-8ede-d6c7bebaec6c admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:51.097200 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:51.097200 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-e41fbf4c-74e7-4afa-8ede-d6c7bebaec6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26a8337ba0804a559c73560e09e223d2', 'request_id': 'req-e41fbf4c-74e7-4afa-8ede-d6c7bebaec6c'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:51.187093 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-e41fbf4c-74e7-4afa-8ede-d6c7bebaec6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26a8337ba0804a559c73560e09e223d2', 'request_id': 'req-e41fbf4c-74e7-4afa-8ede-d6c7bebaec6c'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:51.199591 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-e41fbf4c-74e7-4afa-8ede-d6c7bebaec6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26a8337ba0804a559c73560e09e223d2', 'request_id': 'req-e41fbf4c-74e7-4afa-8ede-d6c7bebaec6c'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:51.231005 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-e41fbf4c-74e7-4afa-8ede-d6c7bebaec6c admin admin] The token cache is being invalidate because project 26a8337ba0804a559c73560e09e223d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:27:51.231487 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-e41fbf4c-74e7-4afa-8ede-d6c7bebaec6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:51.233111 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-e41fbf4c-74e7-4afa-8ede-d6c7bebaec6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:51.240071 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 904/1833] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:27:50 2022] DELETE /identity/v3/projects/26a8337ba0804a559c73560e09e223d2 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:27:51.245093 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97cedf3-bfde-4e14-89d6-d5c39de65a60 service neutron] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:51.262856 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97cedf3-bfde-4e14-89d6-d5c39de65a60 service neutron] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:51.389942 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6577f705-a78d-41b3-9287-fd75eb503117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:51.391072 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6577f705-a78d-41b3-9287-fd75eb503117 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:51.391573 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6577f705-a78d-41b3-9287-fd75eb503117 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:51.392003 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6577f705-a78d-41b3-9287-fd75eb503117 admin admin] PATH_INFO: `/v3/users/cb39723a51b5460a888a3a8c7ab0e9ef` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:51.393999 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6577f705-a78d-41b3-9287-fd75eb503117 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb39723a51b5460a888a3a8c7ab0e9ef)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:51.396066 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6577f705-a78d-41b3-9287-fd75eb503117 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:51.513700 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:51.631550 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-6577f705-a78d-41b3-9287-fd75eb503117 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb39723a51b5460a888a3a8c7ab0e9ef'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:51.664016 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-6577f705-a78d-41b3-9287-fd75eb503117 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb39723a51b5460a888a3a8c7ab0e9ef'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:52.200505 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-6577f705-a78d-41b3-9287-fd75eb503117 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb39723a51b5460a888a3a8c7ab0e9ef'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:52.218475 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-6577f705-a78d-41b3-9287-fd75eb503117 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb39723a51b5460a888a3a8c7ab0e9ef'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:52.218475 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-6577f705-a78d-41b3-9287-fd75eb503117 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb39723a51b5460a888a3a8c7ab0e9ef'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:52.223506 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 930/1834] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:27:51 2022] DELETE /identity/v3/users/cb39723a51b5460a888a3a8c7ab0e9ef => generated 0 bytes in 979 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:27:53.342621 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41fbf4c-74e7-4afa-8ede-d6c7bebaec6c admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:53.342621 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-e41fbf4c-74e7-4afa-8ede-d6c7bebaec6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:53.356315 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41fbf4c-74e7-4afa-8ede-d6c7bebaec6c admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:53.431213 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0faf82-de8e-44e8-8328-eb0e4a083eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:53.434438 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0faf82-de8e-44e8-8328-eb0e4a083eab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:53.434438 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0faf82-de8e-44e8-8328-eb0e4a083eab admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:53.434438 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0faf82-de8e-44e8-8328-eb0e4a083eab admin admin] PATH_INFO: `/v3/projects/3b45a0b8f5014fb78e640ba60eea5e58` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:53.434438 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0faf82-de8e-44e8-8328-eb0e4a083eab admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b45a0b8f5014fb78e640ba60eea5e58)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:53.436891 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0faf82-de8e-44e8-8328-eb0e4a083eab admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:53.597449 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-8e0faf82-de8e-44e8-8328-eb0e4a083eab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b45a0b8f5014fb78e640ba60eea5e58', 'request_id': 'req-8e0faf82-de8e-44e8-8328-eb0e4a083eab'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:53.709219 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ee8f8-4ec6-44d2-9cdb-ec9cb0029c95 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:53.710214 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ee8f8-4ec6-44d2-9cdb-ec9cb0029c95 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:53.710569 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ee8f8-4ec6-44d2-9cdb-ec9cb0029c95 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:53.712350 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 931/1835] 10.222.0.23 () {54 vars in 921 bytes} [Tue Jun 21 18:27:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:27:53.716907 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-8e0faf82-de8e-44e8-8328-eb0e4a083eab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b45a0b8f5014fb78e640ba60eea5e58', 'request_id': 'req-8e0faf82-de8e-44e8-8328-eb0e4a083eab'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:53.726518 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a10791-7f0f-4d17-8744-14990ee1b949 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:53.726821 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a10791-7f0f-4d17-8744-14990ee1b949 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:53.727661 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a10791-7f0f-4d17-8744-14990ee1b949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:53.731018 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-8e0faf82-de8e-44e8-8328-eb0e4a083eab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b45a0b8f5014fb78e640ba60eea5e58', 'request_id': 'req-8e0faf82-de8e-44e8-8328-eb0e4a083eab'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:53.749925 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-f0a10791-7f0f-4d17-8744-14990ee1b949 None None] MFA Rules not processed for user `904255dd3c1f406eb10f6fc65f9cd098`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:53.754678 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-8e0faf82-de8e-44e8-8328-eb0e4a083eab admin admin] The token cache is being invalidate because project 3b45a0b8f5014fb78e640ba60eea5e58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:27:53.754678 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-8e0faf82-de8e-44e8-8328-eb0e4a083eab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:53.754958 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-f0a10791-7f0f-4d17-8744-14990ee1b949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:53.755041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-8e0faf82-de8e-44e8-8328-eb0e4a083eab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:53.759229 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 905/1836] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:27:53 2022] DELETE /identity/v3/projects/3b45a0b8f5014fb78e640ba60eea5e58 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:27:53.760907 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0faf82-de8e-44e8-8328-eb0e4a083eab admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:53.762366 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-8e0faf82-de8e-44e8-8328-eb0e4a083eab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:53.854820 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0faf82-de8e-44e8-8328-eb0e4a083eab admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:53.855573 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 932/1837] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Jun 21 18:27:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:27:53.862230 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca6880a-8ed0-406f-9ea5-cd09ab5cd10a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:53.863289 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6880a-8ed0-406f-9ea5-cd09ab5cd10a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:53.863650 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6880a-8ed0-406f-9ea5-cd09ab5cd10a admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:53.864113 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6880a-8ed0-406f-9ea5-cd09ab5cd10a admin admin] PATH_INFO: `/v3/users/7631cfa60df945b88620db9ae9b47955` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:53.865459 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca6880a-8ed0-406f-9ea5-cd09ab5cd10a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7631cfa60df945b88620db9ae9b47955)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:53.868779 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca6880a-8ed0-406f-9ea5-cd09ab5cd10a admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:53.876161 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:53.886645 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a10791-7f0f-4d17-8744-14990ee1b949 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:53.888905 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-f0a10791-7f0f-4d17-8744-14990ee1b949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:53.907797 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a10791-7f0f-4d17-8744-14990ee1b949 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:53.920539 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b708d5-7b0d-4d8a-87f3-4520a4cedae3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904255dd3c1f406eb10f6fc65f9cd098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:53.921525 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b708d5-7b0d-4d8a-87f3-4520a4cedae3 service nova] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:53.922146 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b708d5-7b0d-4d8a-87f3-4520a4cedae3 service nova] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:53.922577 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b708d5-7b0d-4d8a-87f3-4520a4cedae3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:53.934837 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b708d5-7b0d-4d8a-87f3-4520a4cedae3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:53.939914 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b708d5-7b0d-4d8a-87f3-4520a4cedae3 service nova] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:53.953640 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 933/1838] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:27:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:27:54.091457 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-aca6880a-8ed0-406f-9ea5-cd09ab5cd10a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7631cfa60df945b88620db9ae9b47955'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:54.144156 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-aca6880a-8ed0-406f-9ea5-cd09ab5cd10a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7631cfa60df945b88620db9ae9b47955'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:54.145318 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-aca6880a-8ed0-406f-9ea5-cd09ab5cd10a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7631cfa60df945b88620db9ae9b47955'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:54.159278 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-aca6880a-8ed0-406f-9ea5-cd09ab5cd10a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7631cfa60df945b88620db9ae9b47955'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:54.160288 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-aca6880a-8ed0-406f-9ea5-cd09ab5cd10a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7631cfa60df945b88620db9ae9b47955'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:54.282072 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 906/1839] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:27:53 2022] DELETE /identity/v3/users/7631cfa60df945b88620db9ae9b47955 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:27:54.906858 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b708d5-7b0d-4d8a-87f3-4520a4cedae3 service nova] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:54.908306 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-b7b708d5-7b0d-4d8a-87f3-4520a4cedae3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:54.925548 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca6880a-8ed0-406f-9ea5-cd09ab5cd10a admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:54.925548 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-aca6880a-8ed0-406f-9ea5-cd09ab5cd10a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:54.938438 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b708d5-7b0d-4d8a-87f3-4520a4cedae3 service nova] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:55.018526 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca6880a-8ed0-406f-9ea5-cd09ab5cd10a admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:55.024649 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e8ec12-46f4-483c-be26-4e59148ddcad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:55.025564 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e8ec12-46f4-483c-be26-4e59148ddcad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:55.025854 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e8ec12-46f4-483c-be26-4e59148ddcad admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:55.026149 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e8ec12-46f4-483c-be26-4e59148ddcad admin admin] PATH_INFO: `/v3/users/1e417704153046589f97cddadcc530d7` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:55.027577 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e8ec12-46f4-483c-be26-4e59148ddcad admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e417704153046589f97cddadcc530d7)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:55.029691 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e8ec12-46f4-483c-be26-4e59148ddcad admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:55.035461 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49d4643-4b35-4500-84ba-559776bf2be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:55.036675 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d4643-4b35-4500-84ba-559776bf2be1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:55.037142 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d4643-4b35-4500-84ba-559776bf2be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:55.037598 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d4643-4b35-4500-84ba-559776bf2be1 admin admin] PATH_INFO: `/v3/users/646952e2ed8b4f99a7bec0bd4abbda95` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:55.040408 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49d4643-4b35-4500-84ba-559776bf2be1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=646952e2ed8b4f99a7bec0bd4abbda95)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:55.042317 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49d4643-4b35-4500-84ba-559776bf2be1 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:55.074949 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:55.237686 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-c49d4643-4b35-4500-84ba-559776bf2be1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '646952e2ed8b4f99a7bec0bd4abbda95'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:55.250339 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-d1e8ec12-46f4-483c-be26-4e59148ddcad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e417704153046589f97cddadcc530d7'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:55.253386 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-c49d4643-4b35-4500-84ba-559776bf2be1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '646952e2ed8b4f99a7bec0bd4abbda95'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:55.264062 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-d1e8ec12-46f4-483c-be26-4e59148ddcad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e417704153046589f97cddadcc530d7'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:55.264841 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-d1e8ec12-46f4-483c-be26-4e59148ddcad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e417704153046589f97cddadcc530d7'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:55.271620 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-d1e8ec12-46f4-483c-be26-4e59148ddcad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e417704153046589f97cddadcc530d7'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:55.272252 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-d1e8ec12-46f4-483c-be26-4e59148ddcad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e417704153046589f97cddadcc530d7'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:55.389752 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-c49d4643-4b35-4500-84ba-559776bf2be1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '646952e2ed8b4f99a7bec0bd4abbda95'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:55.390437 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-c49d4643-4b35-4500-84ba-559776bf2be1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '646952e2ed8b4f99a7bec0bd4abbda95'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:55.400589 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-c49d4643-4b35-4500-84ba-559776bf2be1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '646952e2ed8b4f99a7bec0bd4abbda95'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:55.411495 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 934/1840] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:27:54 2022] DELETE /identity/v3/users/646952e2ed8b4f99a7bec0bd4abbda95 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:27:55.413475 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49d4643-4b35-4500-84ba-559776bf2be1 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:55.416676 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-c49d4643-4b35-4500-84ba-559776bf2be1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:55.493166 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49d4643-4b35-4500-84ba-559776bf2be1 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:55.499062 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61f5e1a-fb5b-4d2c-82b0-adf18b48d0fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:55.500125 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61f5e1a-fb5b-4d2c-82b0-adf18b48d0fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:55.500420 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61f5e1a-fb5b-4d2c-82b0-adf18b48d0fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:55.500714 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61f5e1a-fb5b-4d2c-82b0-adf18b48d0fc admin admin] PATH_INFO: `/v3/projects/f9be2888eca04455940d6b3775c12062` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:55.501921 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61f5e1a-fb5b-4d2c-82b0-adf18b48d0fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9be2888eca04455940d6b3775c12062)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:55.507837 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61f5e1a-fb5b-4d2c-82b0-adf18b48d0fc admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:55.511330 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 907/1841] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:27:54 2022] DELETE /identity/v3/users/1e417704153046589f97cddadcc530d7 => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:27:55.512690 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd5dd6-b5cb-4642-a3db-22fcb29c6f8a None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:55.512985 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd5dd6-b5cb-4642-a3db-22fcb29c6f8a None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:55.513266 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd5dd6-b5cb-4642-a3db-22fcb29c6f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:55.526556 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:55.538832 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-66dd5dd6-b5cb-4642-a3db-22fcb29c6f8a None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:55.540560 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-66dd5dd6-b5cb-4642-a3db-22fcb29c6f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:55.603356 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 908/1842] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:27:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:27:55.604800 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a391e2-8a91-4b19-a2af-8f1f4608e04b None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:55.605271 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a391e2-8a91-4b19-a2af-8f1f4608e04b None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:55.605464 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a391e2-8a91-4b19-a2af-8f1f4608e04b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:55.624572 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-e9a391e2-8a91-4b19-a2af-8f1f4608e04b None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:55.625798 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-e9a391e2-8a91-4b19-a2af-8f1f4608e04b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:55.634582 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 909/1843] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:27:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:27:55.635595 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a391e2-8a91-4b19-a2af-8f1f4608e04b None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:55.637044 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-e9a391e2-8a91-4b19-a2af-8f1f4608e04b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:55.685868 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-a61f5e1a-fb5b-4d2c-82b0-adf18b48d0fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9be2888eca04455940d6b3775c12062', 'request_id': 'req-a61f5e1a-fb5b-4d2c-82b0-adf18b48d0fc'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:55.692373 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-a61f5e1a-fb5b-4d2c-82b0-adf18b48d0fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9be2888eca04455940d6b3775c12062', 'request_id': 'req-a61f5e1a-fb5b-4d2c-82b0-adf18b48d0fc'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:55.698887 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-a61f5e1a-fb5b-4d2c-82b0-adf18b48d0fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9be2888eca04455940d6b3775c12062', 'request_id': 'req-a61f5e1a-fb5b-4d2c-82b0-adf18b48d0fc'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:55.701045 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a391e2-8a91-4b19-a2af-8f1f4608e04b None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:55.759095 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25712f9a-9598-4560-9808-0c1b2edbc70e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:55.759681 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25712f9a-9598-4560-9808-0c1b2edbc70e service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:55.759967 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25712f9a-9598-4560-9808-0c1b2edbc70e service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:55.760244 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25712f9a-9598-4560-9808-0c1b2edbc70e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:55.818774 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-a61f5e1a-fb5b-4d2c-82b0-adf18b48d0fc admin admin] The token cache is being invalidate because project f9be2888eca04455940d6b3775c12062 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:27:55.819749 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-a61f5e1a-fb5b-4d2c-82b0-adf18b48d0fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:55.821157 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-a61f5e1a-fb5b-4d2c-82b0-adf18b48d0fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:55.824204 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 935/1844] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:27:55 2022] DELETE /identity/v3/projects/f9be2888eca04455940d6b3775c12062 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:27:55.825237 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61f5e1a-fb5b-4d2c-82b0-adf18b48d0fc admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:55.827053 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-a61f5e1a-fb5b-4d2c-82b0-adf18b48d0fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:55.853458 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61f5e1a-fb5b-4d2c-82b0-adf18b48d0fc admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:55.868944 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25712f9a-9598-4560-9808-0c1b2edbc70e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:55.870261 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af363397-59db-4802-b836-a4558572399f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:55.870707 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25712f9a-9598-4560-9808-0c1b2edbc70e service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:55.870952 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af363397-59db-4802-b836-a4558572399f service neutron] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:55.871240 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af363397-59db-4802-b836-a4558572399f service neutron] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:55.871518 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af363397-59db-4802-b836-a4558572399f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:55.874539 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-af363397-59db-4802-b836-a4558572399f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:55.887344 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 910/1845] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:27:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:27:55.889207 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c664b-082d-4b34-b337-13d2fd934639 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:55.889763 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c664b-082d-4b34-b337-13d2fd934639 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:55.890281 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c664b-082d-4b34-b337-13d2fd934639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:55.896771 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af363397-59db-4802-b836-a4558572399f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:55.898570 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af363397-59db-4802-b836-a4558572399f service neutron] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:55.911908 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 936/1846] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:27:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:27:55.915784 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-e38c664b-082d-4b34-b337-13d2fd934639 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:55.917300 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-e38c664b-082d-4b34-b337-13d2fd934639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:55.926994 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 911/1847] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:27:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:27:55.936724 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a0fd2-5f3a-484e-aaed-8212012fc932 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:55.936965 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a0fd2-5f3a-484e-aaed-8212012fc932 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:55.937242 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a0fd2-5f3a-484e-aaed-8212012fc932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:55.959674 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-b16a0fd2-5f3a-484e-aaed-8212012fc932 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:55.960931 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-b16a0fd2-5f3a-484e-aaed-8212012fc932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:55.972459 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 937/1848] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:27:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:27:55.975840 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38c664b-082d-4b34-b337-13d2fd934639 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:55.987119 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38c664b-082d-4b34-b337-13d2fd934639 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:56.013412 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7754ba62-5c8d-40a9-afa2-9fb9675d23f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:56.014293 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7754ba62-5c8d-40a9-afa2-9fb9675d23f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:56.014404 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7754ba62-5c8d-40a9-afa2-9fb9675d23f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:56.014630 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7754ba62-5c8d-40a9-afa2-9fb9675d23f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:56.015370 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7754ba62-5c8d-40a9-afa2-9fb9675d23f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:27:56.015680 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7754ba62-5c8d-40a9-afa2-9fb9675d23f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:56.017548 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7754ba62-5c8d-40a9-afa2-9fb9675d23f1 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:56.047827 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 912/1849] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:27:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:27:56.048120 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16a0fd2-5f3a-484e-aaed-8212012fc932 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:56.064761 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16a0fd2-5f3a-484e-aaed-8212012fc932 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:56.072089 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc850d8d-c205-4e3a-b484-2d7f0e5dd04c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:56.072897 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc850d8d-c205-4e3a-b484-2d7f0e5dd04c admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:56.073157 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc850d8d-c205-4e3a-b484-2d7f0e5dd04c admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:56.073437 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc850d8d-c205-4e3a-b484-2d7f0e5dd04c admin admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:56.074203 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc850d8d-c205-4e3a-b484-2d7f0e5dd04c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:56.078988 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc850d8d-c205-4e3a-b484-2d7f0e5dd04c admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:56.196715 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 938/1850] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:27:56 2022] POST /identity/v3/projects => generated 383 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:27:56.204664 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7754ba62-5c8d-40a9-afa2-9fb9675d23f1 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:56.215070 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7754ba62-5c8d-40a9-afa2-9fb9675d23f1 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:56.280844 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89549fa-5178-45e8-a1e3-cb1c4c1913cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:56.281700 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89549fa-5178-45e8-a1e3-cb1c4c1913cb admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:56.282002 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89549fa-5178-45e8-a1e3-cb1c4c1913cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:56.282269 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89549fa-5178-45e8-a1e3-cb1c4c1913cb admin admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:56.282980 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89549fa-5178-45e8-a1e3-cb1c4c1913cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:56.284957 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89549fa-5178-45e8-a1e3-cb1c4c1913cb admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:56.333029 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:56.364222 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc850d8d-c205-4e3a-b484-2d7f0e5dd04c admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:56.365268 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 913/1851] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:27:56 2022] POST /identity/v3/users => generated 292 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:27:56.377288 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc850d8d-c205-4e3a-b484-2d7f0e5dd04c admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:56.383941 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a81cb8-2dd5-4a67-a708-16efca577e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:56.384679 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a81cb8-2dd5-4a67-a708-16efca577e85 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:56.384968 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a81cb8-2dd5-4a67-a708-16efca577e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:56.385247 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a81cb8-2dd5-4a67-a708-16efca577e85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:56.385999 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a81cb8-2dd5-4a67-a708-16efca577e85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:56.387647 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a81cb8-2dd5-4a67-a708-16efca577e85 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:56.400094 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 939/1852] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:27:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:27:56.405619 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89549fa-5178-45e8-a1e3-cb1c4c1913cb admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:56.418034 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89549fa-5178-45e8-a1e3-cb1c4c1913cb admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:56.424662 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb17af3-05e6-454b-b82b-94df0ab153ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:56.425178 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb17af3-05e6-454b-b82b-94df0ab153ed admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:56.428625 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb17af3-05e6-454b-b82b-94df0ab153ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:56.428625 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb17af3-05e6-454b-b82b-94df0ab153ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:56.428625 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb17af3-05e6-454b-b82b-94df0ab153ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:56.428625 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb17af3-05e6-454b-b82b-94df0ab153ed admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:56.441074 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 914/1853] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:27:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:27:56.446016 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a81cb8-2dd5-4a67-a708-16efca577e85 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:56.457202 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a81cb8-2dd5-4a67-a708-16efca577e85 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:56.462207 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fda6cc-9062-4cdc-ab30-716008a35c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:56.463045 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fda6cc-9062-4cdc-ab30-716008a35c6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:56.463326 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fda6cc-9062-4cdc-ab30-716008a35c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:56.463603 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fda6cc-9062-4cdc-ab30-716008a35c6d admin admin] PATH_INFO: `/v3/projects/66c9c09c19a94a869e4df6d3c6d32c27/users/21e33a1cf7294b92b870e7e89968a9d4/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:56.480659 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fda6cc-9062-4cdc-ab30-716008a35c6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=66c9c09c19a94a869e4df6d3c6d32c27, user_id=21e33a1cf7294b92b870e7e89968a9d4, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:56.483767 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fda6cc-9062-4cdc-ab30-716008a35c6d admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:56.518454 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb17af3-05e6-454b-b82b-94df0ab153ed admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:56.520122 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-fcb17af3-05e6-454b-b82b-94df0ab153ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:56.557819 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 940/1854] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:27:56 2022] PUT /identity/v3/projects/66c9c09c19a94a869e4df6d3c6d32c27/users/21e33a1cf7294b92b870e7e89968a9d4/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:27:56.572220 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad7408f-7f1b-4e56-9dfa-b954a1cefec4 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:56.572546 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad7408f-7f1b-4e56-9dfa-b954a1cefec4 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:56.573333 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad7408f-7f1b-4e56-9dfa-b954a1cefec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:56.581786 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb17af3-05e6-454b-b82b-94df0ab153ed admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:56.593981 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:56.623684 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-2ad7408f-7f1b-4e56-9dfa-b954a1cefec4 None None] MFA Rules not processed for user `21e33a1cf7294b92b870e7e89968a9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:56.625213 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-2ad7408f-7f1b-4e56-9dfa-b954a1cefec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:56.682163 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4199673f-eace-4858-9493-f9f404187e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:56.683355 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199673f-eace-4858-9493-f9f404187e6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:56.683705 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199673f-eace-4858-9493-f9f404187e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:56.684054 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199673f-eace-4858-9493-f9f404187e6e admin admin] PATH_INFO: `/v3/projects/8378f00d14b844a5ac3d20add3fc208a` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:56.685421 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4199673f-eace-4858-9493-f9f404187e6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8378f00d14b844a5ac3d20add3fc208a)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:56.687759 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4199673f-eace-4858-9493-f9f404187e6e admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:56.733611 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 941/1855] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:27:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:27:56.746051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad7408f-7f1b-4e56-9dfa-b954a1cefec4 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:56.747526 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-2ad7408f-7f1b-4e56-9dfa-b954a1cefec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:56.814097 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad7408f-7f1b-4e56-9dfa-b954a1cefec4 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:56.821464 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a4f798-9f4e-4755-a118-c1619b807f1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:56.822138 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a4f798-9f4e-4755-a118-c1619b807f1d service cinder] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:56.822423 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a4f798-9f4e-4755-a118-c1619b807f1d service cinder] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:56.822704 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a4f798-9f4e-4755-a118-c1619b807f1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:56.834207 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a4f798-9f4e-4755-a118-c1619b807f1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:56.835483 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a4f798-9f4e-4755-a118-c1619b807f1d service cinder] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:56.847541 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 942/1856] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:27:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:27:56.848876 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a4f798-9f4e-4755-a118-c1619b807f1d service cinder] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:56.851213 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-73a4f798-9f4e-4755-a118-c1619b807f1d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:56.877107 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-4199673f-eace-4858-9493-f9f404187e6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8378f00d14b844a5ac3d20add3fc208a', 'request_id': 'req-4199673f-eace-4858-9493-f9f404187e6e'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:56.877640 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a4f798-9f4e-4755-a118-c1619b807f1d service cinder] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:56.933353 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:56.948523 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15536c6e-3c27-49e9-a5a2-df0bc3443314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:56.949401 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15536c6e-3c27-49e9-a5a2-df0bc3443314 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:56.949683 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15536c6e-3c27-49e9-a5a2-df0bc3443314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:56.950541 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15536c6e-3c27-49e9-a5a2-df0bc3443314 admin admin] PATH_INFO: `/v3/projects/527104496b274cd5ae43c211cc496d2e` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:56.953611 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15536c6e-3c27-49e9-a5a2-df0bc3443314 admin admin] RBAC: Authorizing `identity:delete_project(project_id=527104496b274cd5ae43c211cc496d2e)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:56.953611 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15536c6e-3c27-49e9-a5a2-df0bc3443314 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:56.978337 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-4199673f-eace-4858-9493-f9f404187e6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8378f00d14b844a5ac3d20add3fc208a', 'request_id': 'req-4199673f-eace-4858-9493-f9f404187e6e'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:56.984617 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-4199673f-eace-4858-9493-f9f404187e6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8378f00d14b844a5ac3d20add3fc208a', 'request_id': 'req-4199673f-eace-4858-9493-f9f404187e6e'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:56.985814 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:56.997974 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-4199673f-eace-4858-9493-f9f404187e6e admin admin] The token cache is being invalidate because project 8378f00d14b844a5ac3d20add3fc208a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:27:56.998257 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-4199673f-eace-4858-9493-f9f404187e6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:56.999194 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-4199673f-eace-4858-9493-f9f404187e6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:57.000943 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 915/1857] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:27:56 2022] DELETE /identity/v3/projects/8378f00d14b844a5ac3d20add3fc208a => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:27:57.005079 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4199673f-eace-4858-9493-f9f404187e6e admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:57.006600 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-4199673f-eace-4858-9493-f9f404187e6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:57.020380 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4199673f-eace-4858-9493-f9f404187e6e admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:57.027951 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7db27b-3345-4b26-8c3d-81341e697e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:57.028899 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7db27b-3345-4b26-8c3d-81341e697e2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:57.029185 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7db27b-3345-4b26-8c3d-81341e697e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:57.029486 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7db27b-3345-4b26-8c3d-81341e697e2e admin admin] PATH_INFO: `/v3/users/78eb338bf15849b3bd58e33927c289bc` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:57.030784 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7db27b-3345-4b26-8c3d-81341e697e2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=78eb338bf15849b3bd58e33927c289bc)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:57.032722 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7db27b-3345-4b26-8c3d-81341e697e2e admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:57.109498 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-15536c6e-3c27-49e9-a5a2-df0bc3443314 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '527104496b274cd5ae43c211cc496d2e', 'request_id': 'req-15536c6e-3c27-49e9-a5a2-df0bc3443314'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:57.120438 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-15536c6e-3c27-49e9-a5a2-df0bc3443314 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '527104496b274cd5ae43c211cc496d2e', 'request_id': 'req-15536c6e-3c27-49e9-a5a2-df0bc3443314'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:57.125601 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-15536c6e-3c27-49e9-a5a2-df0bc3443314 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '527104496b274cd5ae43c211cc496d2e', 'request_id': 'req-15536c6e-3c27-49e9-a5a2-df0bc3443314'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:57.237135 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-9e7db27b-3345-4b26-8c3d-81341e697e2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78eb338bf15849b3bd58e33927c289bc'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:57.237245 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-15536c6e-3c27-49e9-a5a2-df0bc3443314 admin admin] The token cache is being invalidate because project 527104496b274cd5ae43c211cc496d2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:27:57.237245 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-15536c6e-3c27-49e9-a5a2-df0bc3443314 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:57.237695 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-15536c6e-3c27-49e9-a5a2-df0bc3443314 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:57.245045 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 943/1858] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:27:56 2022] DELETE /identity/v3/projects/527104496b274cd5ae43c211cc496d2e => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:27:57.246647 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15536c6e-3c27-49e9-a5a2-df0bc3443314 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:57.249983 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-15536c6e-3c27-49e9-a5a2-df0bc3443314 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:57.253613 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-9e7db27b-3345-4b26-8c3d-81341e697e2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78eb338bf15849b3bd58e33927c289bc'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:57.259997 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-9e7db27b-3345-4b26-8c3d-81341e697e2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78eb338bf15849b3bd58e33927c289bc'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:57.267112 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-9e7db27b-3345-4b26-8c3d-81341e697e2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78eb338bf15849b3bd58e33927c289bc'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:57.267872 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-9e7db27b-3345-4b26-8c3d-81341e697e2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78eb338bf15849b3bd58e33927c289bc'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:57.333156 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15536c6e-3c27-49e9-a5a2-df0bc3443314 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:57.339142 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14caad9c-f8e9-4676-bea5-7f7062bc206a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:57.340430 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14caad9c-f8e9-4676-bea5-7f7062bc206a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:57.340683 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14caad9c-f8e9-4676-bea5-7f7062bc206a admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:57.341035 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14caad9c-f8e9-4676-bea5-7f7062bc206a admin admin] PATH_INFO: `/v3/users/2c89c6d799ad4189996802de776f7217` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:57.355009 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14caad9c-f8e9-4676-bea5-7f7062bc206a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c89c6d799ad4189996802de776f7217)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:57.359805 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14caad9c-f8e9-4676-bea5-7f7062bc206a admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:57.365479 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 916/1859] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:27:57 2022] DELETE /identity/v3/users/78eb338bf15849b3bd58e33927c289bc => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:27:57.616153 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-14caad9c-f8e9-4676-bea5-7f7062bc206a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c89c6d799ad4189996802de776f7217'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:57.647998 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-14caad9c-f8e9-4676-bea5-7f7062bc206a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c89c6d799ad4189996802de776f7217'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:57.750468 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-14caad9c-f8e9-4676-bea5-7f7062bc206a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c89c6d799ad4189996802de776f7217'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:57.751377 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-14caad9c-f8e9-4676-bea5-7f7062bc206a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c89c6d799ad4189996802de776f7217'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:57.761242 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:57.763952 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-14caad9c-f8e9-4676-bea5-7f7062bc206a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c89c6d799ad4189996802de776f7217'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:57.778469 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 944/1860] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:27:57 2022] DELETE /identity/v3/users/2c89c6d799ad4189996802de776f7217 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:27:58.007993 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7db27b-3345-4b26-8c3d-81341e697e2e admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:58.012096 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-9e7db27b-3345-4b26-8c3d-81341e697e2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:58.128769 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7db27b-3345-4b26-8c3d-81341e697e2e admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:58.233544 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de7ac6b-4148-42f2-8f4d-a3b6c029061d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:58.234451 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de7ac6b-4148-42f2-8f4d-a3b6c029061d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:58.234738 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de7ac6b-4148-42f2-8f4d-a3b6c029061d admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:58.235014 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de7ac6b-4148-42f2-8f4d-a3b6c029061d admin admin] PATH_INFO: `/v3/projects/89a007f9ce194dadb2dba2850c38d11d` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:58.236579 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de7ac6b-4148-42f2-8f4d-a3b6c029061d admin admin] RBAC: Authorizing `identity:delete_project(project_id=89a007f9ce194dadb2dba2850c38d11d)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:58.240116 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de7ac6b-4148-42f2-8f4d-a3b6c029061d admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:58.262869 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:58.363204 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-8de7ac6b-4148-42f2-8f4d-a3b6c029061d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89a007f9ce194dadb2dba2850c38d11d', 'request_id': 'req-8de7ac6b-4148-42f2-8f4d-a3b6c029061d'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:58.429647 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-8de7ac6b-4148-42f2-8f4d-a3b6c029061d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89a007f9ce194dadb2dba2850c38d11d', 'request_id': 'req-8de7ac6b-4148-42f2-8f4d-a3b6c029061d'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:58.439472 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-8de7ac6b-4148-42f2-8f4d-a3b6c029061d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89a007f9ce194dadb2dba2850c38d11d', 'request_id': 'req-8de7ac6b-4148-42f2-8f4d-a3b6c029061d'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:58.458873 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-8de7ac6b-4148-42f2-8f4d-a3b6c029061d admin admin] The token cache is being invalidate because project 89a007f9ce194dadb2dba2850c38d11d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:27:58.459193 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-8de7ac6b-4148-42f2-8f4d-a3b6c029061d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:58.459815 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-8de7ac6b-4148-42f2-8f4d-a3b6c029061d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:58.461651 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 917/1861] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:27:58 2022] DELETE /identity/v3/projects/89a007f9ce194dadb2dba2850c38d11d => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:27:58.465942 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14caad9c-f8e9-4676-bea5-7f7062bc206a admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:58.473172 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-14caad9c-f8e9-4676-bea5-7f7062bc206a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:58.481123 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14caad9c-f8e9-4676-bea5-7f7062bc206a admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:58.545495 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f207ae-64b3-4b6e-867e-7598be8f57ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:58.546401 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f207ae-64b3-4b6e-867e-7598be8f57ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:58.546701 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f207ae-64b3-4b6e-867e-7598be8f57ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:58.547368 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f207ae-64b3-4b6e-867e-7598be8f57ba admin admin] PATH_INFO: `/v3/users/7477c79e28c648ff85d1178d5177e832` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:58.548343 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f207ae-64b3-4b6e-867e-7598be8f57ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=7477c79e28c648ff85d1178d5177e832)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:58.550732 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f207ae-64b3-4b6e-867e-7598be8f57ba admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:58.564696 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de7ac6b-4148-42f2-8f4d-a3b6c029061d admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:58.566337 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-8de7ac6b-4148-42f2-8f4d-a3b6c029061d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:58.612489 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de7ac6b-4148-42f2-8f4d-a3b6c029061d admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:58.620232 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f23d24-f9a8-4b65-a9bd-957be84618ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:58.621165 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f23d24-f9a8-4b65-a9bd-957be84618ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:58.621461 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f23d24-f9a8-4b65-a9bd-957be84618ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:58.621741 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f23d24-f9a8-4b65-a9bd-957be84618ea admin admin] PATH_INFO: `/v3/projects/45237ca4c8b74688ae0b4643727a5240` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:58.623155 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f23d24-f9a8-4b65-a9bd-957be84618ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=45237ca4c8b74688ae0b4643727a5240)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:58.628433 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f23d24-f9a8-4b65-a9bd-957be84618ea admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:58.753869 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-a2f207ae-64b3-4b6e-867e-7598be8f57ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7477c79e28c648ff85d1178d5177e832'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:58.754912 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-e6f23d24-f9a8-4b65-a9bd-957be84618ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45237ca4c8b74688ae0b4643727a5240', 'request_id': 'req-e6f23d24-f9a8-4b65-a9bd-957be84618ea'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:58.768901 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-a2f207ae-64b3-4b6e-867e-7598be8f57ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7477c79e28c648ff85d1178d5177e832'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:58.835433 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-e6f23d24-f9a8-4b65-a9bd-957be84618ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45237ca4c8b74688ae0b4643727a5240', 'request_id': 'req-e6f23d24-f9a8-4b65-a9bd-957be84618ea'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:58.842051 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-e6f23d24-f9a8-4b65-a9bd-957be84618ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45237ca4c8b74688ae0b4643727a5240', 'request_id': 'req-e6f23d24-f9a8-4b65-a9bd-957be84618ea'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:58.855434 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-e6f23d24-f9a8-4b65-a9bd-957be84618ea admin admin] The token cache is being invalidate because project 45237ca4c8b74688ae0b4643727a5240 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:27:58.855721 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-e6f23d24-f9a8-4b65-a9bd-957be84618ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:58.856594 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-e6f23d24-f9a8-4b65-a9bd-957be84618ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:58.863538 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 918/1862] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:27:58 2022] DELETE /identity/v3/projects/45237ca4c8b74688ae0b4643727a5240 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:27:58.865093 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491981f9-4bc9-4532-b5db-f6a7f1f6c375 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:58.865390 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491981f9-4bc9-4532-b5db-f6a7f1f6c375 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:58.865669 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491981f9-4bc9-4532-b5db-f6a7f1f6c375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:58.885834 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-491981f9-4bc9-4532-b5db-f6a7f1f6c375 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:58.886978 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-491981f9-4bc9-4532-b5db-f6a7f1f6c375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:58.939424 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-a2f207ae-64b3-4b6e-867e-7598be8f57ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7477c79e28c648ff85d1178d5177e832'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:58.940606 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-a2f207ae-64b3-4b6e-867e-7598be8f57ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7477c79e28c648ff85d1178d5177e832'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:58.952761 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-a2f207ae-64b3-4b6e-867e-7598be8f57ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7477c79e28c648ff85d1178d5177e832'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:58.959762 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 919/1863] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:27:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:27:58.962063 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b2f802-77c4-4fba-8b94-d1d679cca920 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:58.962435 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b2f802-77c4-4fba-8b94-d1d679cca920 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:58.962787 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b2f802-77c4-4fba-8b94-d1d679cca920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:58.963325 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 945/1864] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:27:58 2022] DELETE /identity/v3/users/7477c79e28c648ff85d1178d5177e832 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:27:58.984929 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-b6b2f802-77c4-4fba-8b94-d1d679cca920 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:58.986245 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-b6b2f802-77c4-4fba-8b94-d1d679cca920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:59.019068 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:59.074258 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 920/1865] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:27:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:27:59.078428 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f207ae-64b3-4b6e-867e-7598be8f57ba admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:59.093358 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f207ae-64b3-4b6e-867e-7598be8f57ba admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:59.104883 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7842af1d-83ee-45bb-98ff-028eee8cbb46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:59.105600 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7842af1d-83ee-45bb-98ff-028eee8cbb46 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:59.105866 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7842af1d-83ee-45bb-98ff-028eee8cbb46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:59.106147 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7842af1d-83ee-45bb-98ff-028eee8cbb46 admin admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:59.111896 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7842af1d-83ee-45bb-98ff-028eee8cbb46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:27:59.111896 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7842af1d-83ee-45bb-98ff-028eee8cbb46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:59.111896 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7842af1d-83ee-45bb-98ff-028eee8cbb46 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:59.123133 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 946/1866] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:27:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:27:59.127652 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b2f802-77c4-4fba-8b94-d1d679cca920 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:59.158146 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b2f802-77c4-4fba-8b94-d1d679cca920 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:59.191335 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2cada9-9fe6-40b8-ba6e-3c2da21c6c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:59.192038 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2cada9-9fe6-40b8-ba6e-3c2da21c6c81 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:59.192327 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2cada9-9fe6-40b8-ba6e-3c2da21c6c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:59.192639 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2cada9-9fe6-40b8-ba6e-3c2da21c6c81 admin admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:59.193524 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2cada9-9fe6-40b8-ba6e-3c2da21c6c81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:59.196187 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2cada9-9fe6-40b8-ba6e-3c2da21c6c81 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:59.367576 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 921/1867] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:27:59 2022] POST /identity/v3/projects => generated 375 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:27:59.368568 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7842af1d-83ee-45bb-98ff-028eee8cbb46 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:59.386582 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7842af1d-83ee-45bb-98ff-028eee8cbb46 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:59.484472 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66a93f0-d975-4754-8d9a-db9dc2603b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:59.488272 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66a93f0-d975-4754-8d9a-db9dc2603b1c admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:59.488846 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66a93f0-d975-4754-8d9a-db9dc2603b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:59.489278 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66a93f0-d975-4754-8d9a-db9dc2603b1c admin admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:59.491843 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66a93f0-d975-4754-8d9a-db9dc2603b1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:59.495429 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66a93f0-d975-4754-8d9a-db9dc2603b1c admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:59.510739 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:27:59.541872 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2cada9-9fe6-40b8-ba6e-3c2da21c6c81 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:59.543625 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-ed2cada9-9fe6-40b8-ba6e-3c2da21c6c81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:59.571295 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2cada9-9fe6-40b8-ba6e-3c2da21c6c81 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:59.576513 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cc7482-b2d5-4568-b2c2-d26407abb765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:59.577408 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc7482-b2d5-4568-b2c2-d26407abb765 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:59.577731 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc7482-b2d5-4568-b2c2-d26407abb765 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:59.577955 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc7482-b2d5-4568-b2c2-d26407abb765 admin admin] PATH_INFO: `/v3/projects/456420aac8ec4e02a08b07824f235417` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:59.580525 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cc7482-b2d5-4568-b2c2-d26407abb765 admin admin] RBAC: Authorizing `identity:delete_project(project_id=456420aac8ec4e02a08b07824f235417)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:59.582522 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cc7482-b2d5-4568-b2c2-d26407abb765 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:59.636658 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 947/1868] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:27:59 2022] POST /identity/v3/users => generated 288 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:27:59.640538 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66a93f0-d975-4754-8d9a-db9dc2603b1c admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:59.655638 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66a93f0-d975-4754-8d9a-db9dc2603b1c admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:59.661695 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b13006-d724-463b-a844-293a877332a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:59.663002 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b13006-d724-463b-a844-293a877332a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:59.663286 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b13006-d724-463b-a844-293a877332a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:59.663614 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b13006-d724-463b-a844-293a877332a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:59.665007 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b13006-d724-463b-a844-293a877332a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:59.666879 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b13006-d724-463b-a844-293a877332a6 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:59.687201 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 948/1869] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:27:59 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:27:59.687201 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b13006-d724-463b-a844-293a877332a6 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:59.696891 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b13006-d724-463b-a844-293a877332a6 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:59.701996 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2958b58-809a-4c6e-9618-002370d04bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:59.702711 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2958b58-809a-4c6e-9618-002370d04bb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:59.702988 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2958b58-809a-4c6e-9618-002370d04bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:59.703259 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2958b58-809a-4c6e-9618-002370d04bb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:59.704009 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2958b58-809a-4c6e-9618-002370d04bb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:59.705665 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2958b58-809a-4c6e-9618-002370d04bb6 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:59.716460 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 949/1870] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:27:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:27:59.720728 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2958b58-809a-4c6e-9618-002370d04bb6 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:27:59.732256 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2958b58-809a-4c6e-9618-002370d04bb6 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:27:59.740365 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0103d694-c906-4b44-98c4-1ffed341a295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:27:59.741417 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0103d694-c906-4b44-98c4-1ffed341a295 admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:59.741741 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0103d694-c906-4b44-98c4-1ffed341a295 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:59.742044 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0103d694-c906-4b44-98c4-1ffed341a295 admin admin] PATH_INFO: `/v3/projects/250848114c81496bab02299e94a26f02/users/0548f4595850464a8e3d9c9b4f7014cf/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:59.779530 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0103d694-c906-4b44-98c4-1ffed341a295 admin admin] RBAC: Authorizing `identity:create_grant(project_id=250848114c81496bab02299e94a26f02, user_id=0548f4595850464a8e3d9c9b4f7014cf, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:27:59.782642 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0103d694-c906-4b44-98c4-1ffed341a295 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:27:59.786353 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-86cc7482-b2d5-4568-b2c2-d26407abb765 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '456420aac8ec4e02a08b07824f235417', 'request_id': 'req-86cc7482-b2d5-4568-b2c2-d26407abb765'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:59.938721 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 950/1871] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:27:59 2022] PUT /identity/v3/projects/250848114c81496bab02299e94a26f02/users/0548f4595850464a8e3d9c9b4f7014cf/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:27:59.949445 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e315db9f-c22c-47a9-8266-7a068de1ffe6 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:27:59.949694 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e315db9f-c22c-47a9-8266-7a068de1ffe6 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:27:59.949975 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e315db9f-c22c-47a9-8266-7a068de1ffe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:27:59.978805 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-e315db9f-c22c-47a9-8266-7a068de1ffe6 None None] MFA Rules not processed for user `0548f4595850464a8e3d9c9b4f7014cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:27:59.981923 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-86cc7482-b2d5-4568-b2c2-d26407abb765 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '456420aac8ec4e02a08b07824f235417', 'request_id': 'req-86cc7482-b2d5-4568-b2c2-d26407abb765'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:27:59.982334 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-e315db9f-c22c-47a9-8266-7a068de1ffe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:27:59.988163 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-86cc7482-b2d5-4568-b2c2-d26407abb765 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '456420aac8ec4e02a08b07824f235417', 'request_id': 'req-86cc7482-b2d5-4568-b2c2-d26407abb765'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:00.006819 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-86cc7482-b2d5-4568-b2c2-d26407abb765 admin admin] The token cache is being invalidate because project 456420aac8ec4e02a08b07824f235417 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:28:00.007251 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-86cc7482-b2d5-4568-b2c2-d26407abb765 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:00.008467 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-86cc7482-b2d5-4568-b2c2-d26407abb765 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:00.010122 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 922/1872] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:27:59 2022] DELETE /identity/v3/projects/456420aac8ec4e02a08b07824f235417 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:28:00.016022 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e8346-6654-4c36-bd2f-466aa5af8ed5 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:00.016365 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e8346-6654-4c36-bd2f-466aa5af8ed5 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:00.016702 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e8346-6654-4c36-bd2f-466aa5af8ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:00.037929 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-414e8346-6654-4c36-bd2f-466aa5af8ed5 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:00.039166 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-414e8346-6654-4c36-bd2f-466aa5af8ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:00.051084 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 951/1873] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:27:59 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:28:00.055729 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e315db9f-c22c-47a9-8266-7a068de1ffe6 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:00.057003 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-e315db9f-c22c-47a9-8266-7a068de1ffe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:00.105081 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 923/1874] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:28:00.118079 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610d661f-e16a-41b5-a479-690526db50b6 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:00.118473 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610d661f-e16a-41b5-a479-690526db50b6 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:00.118873 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610d661f-e16a-41b5-a479-690526db50b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:00.128188 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e315db9f-c22c-47a9-8266-7a068de1ffe6 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:00.139894 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5c8542-83cd-4d8a-a4d3-53411a521e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:00.140640 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c8542-83cd-4d8a-a4d3-53411a521e83 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:00.140936 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c8542-83cd-4d8a-a4d3-53411a521e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:00.143801 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c8542-83cd-4d8a-a4d3-53411a521e83 admin admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:00.143801 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5c8542-83cd-4d8a-a4d3-53411a521e83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:00.143968 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5c8542-83cd-4d8a-a4d3-53411a521e83 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:00.144417 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-610d661f-e16a-41b5-a479-690526db50b6 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:00.145684 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-610d661f-e16a-41b5-a479-690526db50b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:00.155108 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 924/1875] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:28:00.159253 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610d661f-e16a-41b5-a479-690526db50b6 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:00.168127 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610d661f-e16a-41b5-a479-690526db50b6 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:00.173555 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0856d143-5f5e-4cda-a39b-5b3086adbd2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:00.174874 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0856d143-5f5e-4cda-a39b-5b3086adbd2f admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:00.175200 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0856d143-5f5e-4cda-a39b-5b3086adbd2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:00.175478 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0856d143-5f5e-4cda-a39b-5b3086adbd2f admin admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:00.176544 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0856d143-5f5e-4cda-a39b-5b3086adbd2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:28:00.176856 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0856d143-5f5e-4cda-a39b-5b3086adbd2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:00.178769 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0856d143-5f5e-4cda-a39b-5b3086adbd2f admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:00.194586 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 925/1876] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:28:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:28:00.195473 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0856d143-5f5e-4cda-a39b-5b3086adbd2f admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:00.206479 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0856d143-5f5e-4cda-a39b-5b3086adbd2f admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:00.212332 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283d48c8-9ce1-4a1b-8061-2e5cf177b4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:00.213102 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283d48c8-9ce1-4a1b-8061-2e5cf177b4a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:00.213399 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283d48c8-9ce1-4a1b-8061-2e5cf177b4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:00.213654 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283d48c8-9ce1-4a1b-8061-2e5cf177b4a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:00.214344 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283d48c8-9ce1-4a1b-8061-2e5cf177b4a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:00.216217 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283d48c8-9ce1-4a1b-8061-2e5cf177b4a2 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:00.225575 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:00.259631 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 952/1877] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:28:00 2022] POST /identity/v3/projects => generated 375 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:28:00.263662 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5c8542-83cd-4d8a-a4d3-53411a521e83 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:00.326031 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5c8542-83cd-4d8a-a4d3-53411a521e83 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:00.336695 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85c6e09-8149-4f1d-b37a-036d058455e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:00.336695 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c6e09-8149-4f1d-b37a-036d058455e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:00.336695 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c6e09-8149-4f1d-b37a-036d058455e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:00.336695 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c6e09-8149-4f1d-b37a-036d058455e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:00.337017 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85c6e09-8149-4f1d-b37a-036d058455e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:00.337017 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85c6e09-8149-4f1d-b37a-036d058455e6 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:00.347229 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 926/1878] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:28:00 2022] POST /identity/v3/projects => generated 385 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:28:00.355879 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283d48c8-9ce1-4a1b-8061-2e5cf177b4a2 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:00.419870 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 953/1879] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:28:00 2022] POST /identity/v3/users => generated 294 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:28:00.421102 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283d48c8-9ce1-4a1b-8061-2e5cf177b4a2 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:00.424000 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85c6e09-8149-4f1d-b37a-036d058455e6 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:00.427085 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecf776b-1a47-4964-a7ba-e6af5c0b2d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:00.427877 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecf776b-1a47-4964-a7ba-e6af5c0b2d6b admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:00.428132 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecf776b-1a47-4964-a7ba-e6af5c0b2d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:00.428441 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecf776b-1a47-4964-a7ba-e6af5c0b2d6b admin admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:00.429186 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecf776b-1a47-4964-a7ba-e6af5c0b2d6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:00.432084 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecf776b-1a47-4964-a7ba-e6af5c0b2d6b admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:00.438940 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85c6e09-8149-4f1d-b37a-036d058455e6 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:00.444125 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b904b828-6821-4741-b6a1-0d10ffd91b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:00.444910 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b904b828-6821-4741-b6a1-0d10ffd91b1b admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:00.445214 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b904b828-6821-4741-b6a1-0d10ffd91b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:00.445421 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b904b828-6821-4741-b6a1-0d10ffd91b1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:00.446201 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b904b828-6821-4741-b6a1-0d10ffd91b1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:00.447900 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b904b828-6821-4741-b6a1-0d10ffd91b1b admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:00.459523 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 954/1880] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:28:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:28:00.463616 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b904b828-6821-4741-b6a1-0d10ffd91b1b admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:00.476508 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b904b828-6821-4741-b6a1-0d10ffd91b1b admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:00.482917 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d81ee4-6990-4328-81d6-17d22c6c9728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:00.483887 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d81ee4-6990-4328-81d6-17d22c6c9728 admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:00.484248 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d81ee4-6990-4328-81d6-17d22c6c9728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:00.484505 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d81ee4-6990-4328-81d6-17d22c6c9728 admin admin] PATH_INFO: `/v3/projects/7a96532be6ee4f1ab16e6e3024fa1e9f/users/88452e8587964fae866fb6014424a170/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:00.501431 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d81ee4-6990-4328-81d6-17d22c6c9728 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a96532be6ee4f1ab16e6e3024fa1e9f, user_id=88452e8587964fae866fb6014424a170, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:00.504147 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d81ee4-6990-4328-81d6-17d22c6c9728 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:00.515170 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 927/1881] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:28:00 2022] POST /identity/v3/users => generated 293 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:28:00.515170 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecf776b-1a47-4964-a7ba-e6af5c0b2d6b admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:00.527940 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecf776b-1a47-4964-a7ba-e6af5c0b2d6b admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:00.533136 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c035f0d4-2753-4720-b31d-8156fedddaa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:00.544086 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c035f0d4-2753-4720-b31d-8156fedddaa0 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:00.544086 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c035f0d4-2753-4720-b31d-8156fedddaa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:00.544086 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c035f0d4-2753-4720-b31d-8156fedddaa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:00.544086 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c035f0d4-2753-4720-b31d-8156fedddaa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:00.544086 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c035f0d4-2753-4720-b31d-8156fedddaa0 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:00.548423 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 928/1882] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:28:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:28:00.552228 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c035f0d4-2753-4720-b31d-8156fedddaa0 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:00.563140 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c035f0d4-2753-4720-b31d-8156fedddaa0 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:00.568219 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c20c1e-3f41-4662-9093-f50084b1d91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:00.568937 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c20c1e-3f41-4662-9093-f50084b1d91e admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:00.569226 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c20c1e-3f41-4662-9093-f50084b1d91e admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:00.569507 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c20c1e-3f41-4662-9093-f50084b1d91e admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:00.570245 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c20c1e-3f41-4662-9093-f50084b1d91e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:00.572106 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c20c1e-3f41-4662-9093-f50084b1d91e admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:00.584511 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 929/1883] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:28:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:28:00.589606 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c20c1e-3f41-4662-9093-f50084b1d91e admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:00.603549 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 955/1884] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:28:00 2022] PUT /identity/v3/projects/7a96532be6ee4f1ab16e6e3024fa1e9f/users/88452e8587964fae866fb6014424a170/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:00.607448 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d81ee4-6990-4328-81d6-17d22c6c9728 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:00.611073 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c20c1e-3f41-4662-9093-f50084b1d91e admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:00.670729 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5b8470-8c79-4a0c-b2e9-a63456bac194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:00.671596 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b8470-8c79-4a0c-b2e9-a63456bac194 admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:00.671879 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b8470-8c79-4a0c-b2e9-a63456bac194 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:00.672153 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b8470-8c79-4a0c-b2e9-a63456bac194 admin admin] PATH_INFO: `/v3/projects/7036fc0268144b39ae5e78597e8e8c15/users/3a2ecd910e344135a8dd33b5c2d818b8/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:00.672368 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d81ee4-6990-4328-81d6-17d22c6c9728 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:00.677607 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ff4160-2c86-4ff5-8457-2c60a2d2d7b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:00.678285 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ff4160-2c86-4ff5-8457-2c60a2d2d7b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:00.678576 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ff4160-2c86-4ff5-8457-2c60a2d2d7b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:00.678973 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ff4160-2c86-4ff5-8457-2c60a2d2d7b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:00.679881 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ff4160-2c86-4ff5-8457-2c60a2d2d7b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:00.681570 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ff4160-2c86-4ff5-8457-2c60a2d2d7b9 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:00.688132 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5b8470-8c79-4a0c-b2e9-a63456bac194 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7036fc0268144b39ae5e78597e8e8c15, user_id=3a2ecd910e344135a8dd33b5c2d818b8, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:00.690879 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5b8470-8c79-4a0c-b2e9-a63456bac194 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:00.695464 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 956/1885] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:28:00 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:28:00.696781 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ff4160-2c86-4ff5-8457-2c60a2d2d7b9 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:00.706414 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ff4160-2c86-4ff5-8457-2c60a2d2d7b9 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:00.711695 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205d1937-c674-4022-86ea-6f81362b5619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:00.712723 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d1937-c674-4022-86ea-6f81362b5619 admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:00.713011 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d1937-c674-4022-86ea-6f81362b5619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:00.713293 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d1937-c674-4022-86ea-6f81362b5619 admin admin] PATH_INFO: `/v3/projects/7a96532be6ee4f1ab16e6e3024fa1e9f/users/88452e8587964fae866fb6014424a170/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:00.715325 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205d1937-c674-4022-86ea-6f81362b5619 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a96532be6ee4f1ab16e6e3024fa1e9f, user_id=88452e8587964fae866fb6014424a170, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:00.717991 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205d1937-c674-4022-86ea-6f81362b5619 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:00.728954 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 957/1886] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:28:00 2022] PUT /identity/v3/projects/7a96532be6ee4f1ab16e6e3024fa1e9f/users/88452e8587964fae866fb6014424a170/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:00.732458 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:00.740097 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a6dcc-20df-4c77-8799-8442d68b60de None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:00.740412 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a6dcc-20df-4c77-8799-8442d68b60de None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:00.740727 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a6dcc-20df-4c77-8799-8442d68b60de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:00.769128 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-535a6dcc-20df-4c77-8799-8442d68b60de None None] MFA Rules not processed for user `88452e8587964fae866fb6014424a170`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:00.769544 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 930/1887] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:28:00 2022] PUT /identity/v3/projects/7036fc0268144b39ae5e78597e8e8c15/users/3a2ecd910e344135a8dd33b5c2d818b8/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:28:00.770363 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-535a6dcc-20df-4c77-8799-8442d68b60de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:00.781627 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f401d-94df-4680-b7dc-3c76390371d2 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:00.781997 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f401d-94df-4680-b7dc-3c76390371d2 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:00.782282 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f401d-94df-4680-b7dc-3c76390371d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:00.817781 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-0b2f401d-94df-4680-b7dc-3c76390371d2 None None] MFA Rules not processed for user `3a2ecd910e344135a8dd33b5c2d818b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:00.819328 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-0b2f401d-94df-4680-b7dc-3c76390371d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:00.858719 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 958/1888] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:00 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:28:00.870929 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535a6dcc-20df-4c77-8799-8442d68b60de None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:00.872779 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-535a6dcc-20df-4c77-8799-8442d68b60de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:00.934455 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 931/1889] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:28:00.940733 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2f401d-94df-4680-b7dc-3c76390371d2 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:00.950311 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535a6dcc-20df-4c77-8799-8442d68b60de None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:00.954716 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12a7c8a-7ee4-4c22-a13c-e3dd8ee09123 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:00.955417 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a7c8a-7ee4-4c22-a13c-e3dd8ee09123 service cinder] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:00.955706 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a7c8a-7ee4-4c22-a13c-e3dd8ee09123 service cinder] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:00.955985 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a7c8a-7ee4-4c22-a13c-e3dd8ee09123 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:01.017881 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2f401d-94df-4680-b7dc-3c76390371d2 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:01.030307 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58b3b1f-218e-4e4e-bdd7-3de00bab9ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:01.030307 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58b3b1f-218e-4e4e-bdd7-3de00bab9ec4 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:01.030307 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58b3b1f-218e-4e4e-bdd7-3de00bab9ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:01.030307 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58b3b1f-218e-4e4e-bdd7-3de00bab9ec4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:01.030658 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58b3b1f-218e-4e4e-bdd7-3de00bab9ec4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:01.030658 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58b3b1f-218e-4e4e-bdd7-3de00bab9ec4 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:01.034019 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12a7c8a-7ee4-4c22-a13c-e3dd8ee09123 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:01.035492 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12a7c8a-7ee4-4c22-a13c-e3dd8ee09123 service cinder] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:01.045742 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 959/1890] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:28:00 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:28:01.151844 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 932/1891] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:28:00 2022] POST /identity/v3/projects => generated 385 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:28:01.156025 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12a7c8a-7ee4-4c22-a13c-e3dd8ee09123 service cinder] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:01.168260 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12a7c8a-7ee4-4c22-a13c-e3dd8ee09123 service cinder] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:01.173045 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:01.225223 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22043da-e56a-460e-8301-7c01cce01941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:01.226103 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22043da-e56a-460e-8301-7c01cce01941 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:01.226103 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22043da-e56a-460e-8301-7c01cce01941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:01.226360 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22043da-e56a-460e-8301-7c01cce01941 admin admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:01.227916 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22043da-e56a-460e-8301-7c01cce01941 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:01.229982 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22043da-e56a-460e-8301-7c01cce01941 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:01.287014 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:01.344464 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 960/1892] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:28:01 2022] POST /identity/v3/users => generated 299 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:28:01.348424 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58b3b1f-218e-4e4e-bdd7-3de00bab9ec4 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:01.359660 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58b3b1f-218e-4e4e-bdd7-3de00bab9ec4 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:01.364430 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bae1f7-0af0-4e66-b5c1-24aa7e96d4f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:01.365191 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bae1f7-0af0-4e66-b5c1-24aa7e96d4f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:01.365443 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bae1f7-0af0-4e66-b5c1-24aa7e96d4f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:01.365725 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bae1f7-0af0-4e66-b5c1-24aa7e96d4f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:01.366461 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bae1f7-0af0-4e66-b5c1-24aa7e96d4f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:01.368069 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bae1f7-0af0-4e66-b5c1-24aa7e96d4f3 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:01.379586 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 933/1893] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:28:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:28:01.383811 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22043da-e56a-460e-8301-7c01cce01941 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:01.394516 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22043da-e56a-460e-8301-7c01cce01941 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:01.400292 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c5c9ef-9834-4364-8a55-fd960e406be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:01.401246 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c5c9ef-9834-4364-8a55-fd960e406be3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:01.401533 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c5c9ef-9834-4364-8a55-fd960e406be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:01.401811 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c5c9ef-9834-4364-8a55-fd960e406be3 admin admin] PATH_INFO: `/v3/projects/ffa2f6502ea44fcca22a99622f1d37f5/users/0649c7ad928b46ea9b4cf55b3149e873/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:01.417702 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c5c9ef-9834-4364-8a55-fd960e406be3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffa2f6502ea44fcca22a99622f1d37f5, user_id=0649c7ad928b46ea9b4cf55b3149e873, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:01.420343 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c5c9ef-9834-4364-8a55-fd960e406be3 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:01.515646 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 961/1894] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:28:01 2022] PUT /identity/v3/projects/ffa2f6502ea44fcca22a99622f1d37f5/users/0649c7ad928b46ea9b4cf55b3149e873/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:01.519579 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bae1f7-0af0-4e66-b5c1-24aa7e96d4f3 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:01.530332 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bae1f7-0af0-4e66-b5c1-24aa7e96d4f3 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:01.588142 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d50a5c-de53-4d87-a19b-f395b5b79240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:01.588856 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d50a5c-de53-4d87-a19b-f395b5b79240 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:01.589149 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d50a5c-de53-4d87-a19b-f395b5b79240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:01.589428 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d50a5c-de53-4d87-a19b-f395b5b79240 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:01.590155 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d50a5c-de53-4d87-a19b-f395b5b79240 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:01.591808 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d50a5c-de53-4d87-a19b-f395b5b79240 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:01.603165 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 934/1895] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:28:01 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:28:01.607301 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c5c9ef-9834-4364-8a55-fd960e406be3 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:01.618307 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c5c9ef-9834-4364-8a55-fd960e406be3 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:01.623043 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37069f01-1bf9-452d-9771-25127d602b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:01.624198 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37069f01-1bf9-452d-9771-25127d602b14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:01.624515 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37069f01-1bf9-452d-9771-25127d602b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:01.624798 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37069f01-1bf9-452d-9771-25127d602b14 admin admin] PATH_INFO: `/v3/projects/ffa2f6502ea44fcca22a99622f1d37f5/users/0649c7ad928b46ea9b4cf55b3149e873/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:01.626783 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37069f01-1bf9-452d-9771-25127d602b14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffa2f6502ea44fcca22a99622f1d37f5, user_id=0649c7ad928b46ea9b4cf55b3149e873, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:01.629569 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37069f01-1bf9-452d-9771-25127d602b14 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:01.641661 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 962/1896] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:28:01 2022] PUT /identity/v3/projects/ffa2f6502ea44fcca22a99622f1d37f5/users/0649c7ad928b46ea9b4cf55b3149e873/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:01.655678 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a204da64-c2ab-4c32-9025-b64556e41612 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:01.656004 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a204da64-c2ab-4c32-9025-b64556e41612 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:01.656254 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a204da64-c2ab-4c32-9025-b64556e41612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:01.684476 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-a204da64-c2ab-4c32-9025-b64556e41612 None None] MFA Rules not processed for user `0649c7ad928b46ea9b4cf55b3149e873`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:01.685717 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-a204da64-c2ab-4c32-9025-b64556e41612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:01.715314 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:01.761453 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:01.766439 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 935/1897] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:01 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:28:01.793187 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37069f01-1bf9-452d-9771-25127d602b14 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:01.844675 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37069f01-1bf9-452d-9771-25127d602b14 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:01.848536 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a9e2e3-934a-4f82-8bf5-c269a35bb03a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:01.849194 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a9e2e3-934a-4f82-8bf5-c269a35bb03a service cinder] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:01.849472 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a9e2e3-934a-4f82-8bf5-c269a35bb03a service cinder] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:01.849744 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a9e2e3-934a-4f82-8bf5-c269a35bb03a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:01.861239 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a9e2e3-934a-4f82-8bf5-c269a35bb03a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:01.862780 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a9e2e3-934a-4f82-8bf5-c269a35bb03a service cinder] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:01.873056 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 963/1898] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:28:01 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:28:03.368288 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a204da64-c2ab-4c32-9025-b64556e41612 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:03.463591 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a204da64-c2ab-4c32-9025-b64556e41612 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:03.468612 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95424b31-9cd1-4c89-943e-6615f8bef83a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:03.469501 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95424b31-9cd1-4c89-943e-6615f8bef83a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:03.469792 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95424b31-9cd1-4c89-943e-6615f8bef83a admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:03.470074 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95424b31-9cd1-4c89-943e-6615f8bef83a admin admin] PATH_INFO: `/v3/users/3a2ecd910e344135a8dd33b5c2d818b8` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:03.471299 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95424b31-9cd1-4c89-943e-6615f8bef83a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a2ecd910e344135a8dd33b5c2d818b8)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:03.473264 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95424b31-9cd1-4c89-943e-6615f8bef83a admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:03.658549 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-95424b31-9cd1-4c89-943e-6615f8bef83a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a2ecd910e344135a8dd33b5c2d818b8'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:03.673456 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-95424b31-9cd1-4c89-943e-6615f8bef83a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a2ecd910e344135a8dd33b5c2d818b8'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:03.674174 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-95424b31-9cd1-4c89-943e-6615f8bef83a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a2ecd910e344135a8dd33b5c2d818b8'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:03.682959 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-95424b31-9cd1-4c89-943e-6615f8bef83a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a2ecd910e344135a8dd33b5c2d818b8'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:03.683836 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-95424b31-9cd1-4c89-943e-6615f8bef83a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a2ecd910e344135a8dd33b5c2d818b8'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:03.763605 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 936/1899] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:28:03 2022] DELETE /identity/v3/users/3a2ecd910e344135a8dd33b5c2d818b8 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:03.768298 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db210a94-a618-4be8-8eda-2234e8ae4a3b None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:03.769294 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db210a94-a618-4be8-8eda-2234e8ae4a3b None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:03.769613 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db210a94-a618-4be8-8eda-2234e8ae4a3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:03.787644 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-db210a94-a618-4be8-8eda-2234e8ae4a3b None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:03.788910 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-db210a94-a618-4be8-8eda-2234e8ae4a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:03.866141 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 964/1900] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:28:03.873445 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95424b31-9cd1-4c89-943e-6615f8bef83a admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:03.875154 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-95424b31-9cd1-4c89-943e-6615f8bef83a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:03.935005 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:03.950704 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95424b31-9cd1-4c89-943e-6615f8bef83a admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:03.955274 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faad12b2-adf5-46cd-a3ce-08b48ea00cab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:03.956745 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faad12b2-adf5-46cd-a3ce-08b48ea00cab service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:03.957275 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faad12b2-adf5-46cd-a3ce-08b48ea00cab service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:03.957765 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faad12b2-adf5-46cd-a3ce-08b48ea00cab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:03.974136 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faad12b2-adf5-46cd-a3ce-08b48ea00cab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:03.977058 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faad12b2-adf5-46cd-a3ce-08b48ea00cab service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:03.993247 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 937/1901] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:28:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:28:04.546530 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db210a94-a618-4be8-8eda-2234e8ae4a3b None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:04.547986 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-db210a94-a618-4be8-8eda-2234e8ae4a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:04.561384 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db210a94-a618-4be8-8eda-2234e8ae4a3b None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:04.566467 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f69886c-b1dd-453f-8283-4ea07cac7f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:04.567348 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f69886c-b1dd-453f-8283-4ea07cac7f03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:04.567637 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f69886c-b1dd-453f-8283-4ea07cac7f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:04.568584 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f69886c-b1dd-453f-8283-4ea07cac7f03 admin admin] PATH_INFO: `/v3/projects/7036fc0268144b39ae5e78597e8e8c15` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:04.569886 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f69886c-b1dd-453f-8283-4ea07cac7f03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7036fc0268144b39ae5e78597e8e8c15)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:04.571963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f69886c-b1dd-453f-8283-4ea07cac7f03 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:04.694986 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-6f69886c-b1dd-453f-8283-4ea07cac7f03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7036fc0268144b39ae5e78597e8e8c15', 'request_id': 'req-6f69886c-b1dd-453f-8283-4ea07cac7f03'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:04.702788 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-6f69886c-b1dd-453f-8283-4ea07cac7f03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7036fc0268144b39ae5e78597e8e8c15', 'request_id': 'req-6f69886c-b1dd-453f-8283-4ea07cac7f03'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:04.708590 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-6f69886c-b1dd-453f-8283-4ea07cac7f03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7036fc0268144b39ae5e78597e8e8c15', 'request_id': 'req-6f69886c-b1dd-453f-8283-4ea07cac7f03'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:04.794584 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-6f69886c-b1dd-453f-8283-4ea07cac7f03 admin admin] The token cache is being invalidate because project 7036fc0268144b39ae5e78597e8e8c15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:28:04.795234 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-6f69886c-b1dd-453f-8283-4ea07cac7f03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:04.795480 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-6f69886c-b1dd-453f-8283-4ea07cac7f03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:04.797094 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 965/1902] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:28:04 2022] DELETE /identity/v3/projects/7036fc0268144b39ae5e78597e8e8c15 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:04.801346 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faad12b2-adf5-46cd-a3ce-08b48ea00cab service neutron] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:04.811417 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faad12b2-adf5-46cd-a3ce-08b48ea00cab service neutron] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:04.871276 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ebec10-6908-4e46-a146-b6a91d3e1c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:04.872195 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ebec10-6908-4e46-a146-b6a91d3e1c9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:04.872570 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ebec10-6908-4e46-a146-b6a91d3e1c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:04.872852 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ebec10-6908-4e46-a146-b6a91d3e1c9b admin admin] PATH_INFO: `/v3/users/0649c7ad928b46ea9b4cf55b3149e873` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:04.874433 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ebec10-6908-4e46-a146-b6a91d3e1c9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0649c7ad928b46ea9b4cf55b3149e873)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:04.876380 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ebec10-6908-4e46-a146-b6a91d3e1c9b admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:05.122212 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-52ebec10-6908-4e46-a146-b6a91d3e1c9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0649c7ad928b46ea9b4cf55b3149e873'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:05.134712 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-52ebec10-6908-4e46-a146-b6a91d3e1c9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0649c7ad928b46ea9b4cf55b3149e873'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:05.136038 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-52ebec10-6908-4e46-a146-b6a91d3e1c9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0649c7ad928b46ea9b4cf55b3149e873'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:05.142962 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-52ebec10-6908-4e46-a146-b6a91d3e1c9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0649c7ad928b46ea9b4cf55b3149e873'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:05.143601 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-52ebec10-6908-4e46-a146-b6a91d3e1c9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0649c7ad928b46ea9b4cf55b3149e873'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:05.229587 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 938/1903] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:28:04 2022] DELETE /identity/v3/users/0649c7ad928b46ea9b4cf55b3149e873 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:05.694994 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f69886c-b1dd-453f-8283-4ea07cac7f03 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:05.696489 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-6f69886c-b1dd-453f-8283-4ea07cac7f03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:05.711292 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f69886c-b1dd-453f-8283-4ea07cac7f03 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:05.739070 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:05.795080 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611f8d22-a37f-4276-90c2-4876beed1b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:05.795999 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f8d22-a37f-4276-90c2-4876beed1b51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:05.796282 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f8d22-a37f-4276-90c2-4876beed1b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:05.796590 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f8d22-a37f-4276-90c2-4876beed1b51 admin admin] PATH_INFO: `/v3/projects/ffa2f6502ea44fcca22a99622f1d37f5` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:05.797859 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611f8d22-a37f-4276-90c2-4876beed1b51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffa2f6502ea44fcca22a99622f1d37f5)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:05.799785 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611f8d22-a37f-4276-90c2-4876beed1b51 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:05.944037 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-611f8d22-a37f-4276-90c2-4876beed1b51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffa2f6502ea44fcca22a99622f1d37f5', 'request_id': 'req-611f8d22-a37f-4276-90c2-4876beed1b51'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:05.950369 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-611f8d22-a37f-4276-90c2-4876beed1b51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffa2f6502ea44fcca22a99622f1d37f5', 'request_id': 'req-611f8d22-a37f-4276-90c2-4876beed1b51'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:05.955921 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-611f8d22-a37f-4276-90c2-4876beed1b51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffa2f6502ea44fcca22a99622f1d37f5', 'request_id': 'req-611f8d22-a37f-4276-90c2-4876beed1b51'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:06.052061 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-611f8d22-a37f-4276-90c2-4876beed1b51 admin admin] The token cache is being invalidate because project ffa2f6502ea44fcca22a99622f1d37f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:28:06.052357 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-611f8d22-a37f-4276-90c2-4876beed1b51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:06.053099 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-611f8d22-a37f-4276-90c2-4876beed1b51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:06.054680 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 966/1904] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:28:05 2022] DELETE /identity/v3/projects/ffa2f6502ea44fcca22a99622f1d37f5 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:06.059818 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c863f-b1ed-44bf-8588-65d8d8b5a463 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:06.060280 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c863f-b1ed-44bf-8588-65d8d8b5a463 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:06.060371 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c863f-b1ed-44bf-8588-65d8d8b5a463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:06.078887 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-f34c863f-b1ed-44bf-8588-65d8d8b5a463 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:06.080128 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-f34c863f-b1ed-44bf-8588-65d8d8b5a463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:06.149653 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 939/1905] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:28:06.159050 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc363d-4b5c-4f9c-a4f7-ef6472806f0c None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:06.161281 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc363d-4b5c-4f9c-a4f7-ef6472806f0c None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:06.161281 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc363d-4b5c-4f9c-a4f7-ef6472806f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:06.184371 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-9fbc363d-4b5c-4f9c-a4f7-ef6472806f0c None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:06.184371 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-9fbc363d-4b5c-4f9c-a4f7-ef6472806f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:06.189278 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 967/1906] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:28:06.193737 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c863f-b1ed-44bf-8588-65d8d8b5a463 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:06.199914 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:06.203833 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c863f-b1ed-44bf-8588-65d8d8b5a463 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:06.208369 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb7e411-3a0f-48a9-84c4-2cecf93df69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:06.209054 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb7e411-3a0f-48a9-84c4-2cecf93df69e admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:06.209335 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb7e411-3a0f-48a9-84c4-2cecf93df69e admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:06.209610 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb7e411-3a0f-48a9-84c4-2cecf93df69e admin admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:06.210359 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb7e411-3a0f-48a9-84c4-2cecf93df69e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:28:06.210693 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb7e411-3a0f-48a9-84c4-2cecf93df69e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:06.212282 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb7e411-3a0f-48a9-84c4-2cecf93df69e admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:06.225472 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 940/1907] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:28:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:28:06.230272 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc363d-4b5c-4f9c-a4f7-ef6472806f0c None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:06.239603 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc363d-4b5c-4f9c-a4f7-ef6472806f0c None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:06.244277 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2a84cd-7bc7-4fea-a5da-22c12fbaba91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:06.245703 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2a84cd-7bc7-4fea-a5da-22c12fbaba91 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:06.245990 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2a84cd-7bc7-4fea-a5da-22c12fbaba91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:06.246268 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2a84cd-7bc7-4fea-a5da-22c12fbaba91 admin admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:06.246964 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2a84cd-7bc7-4fea-a5da-22c12fbaba91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:06.249191 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2a84cd-7bc7-4fea-a5da-22c12fbaba91 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:06.359056 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 968/1908] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:28:06 2022] POST /identity/v3/projects => generated 379 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:28:06.361625 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb7e411-3a0f-48a9-84c4-2cecf93df69e admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:06.372188 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb7e411-3a0f-48a9-84c4-2cecf93df69e admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:06.435531 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7556bf22-1683-41a3-843d-1a116a3b11ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:06.436138 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7556bf22-1683-41a3-843d-1a116a3b11ff admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:06.437384 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7556bf22-1683-41a3-843d-1a116a3b11ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:06.437673 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7556bf22-1683-41a3-843d-1a116a3b11ff admin admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:06.438739 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7556bf22-1683-41a3-843d-1a116a3b11ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:06.440705 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7556bf22-1683-41a3-843d-1a116a3b11ff admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:06.554614 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 941/1909] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:28:06 2022] POST /identity/v3/users => generated 290 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:28:06.558993 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2a84cd-7bc7-4fea-a5da-22c12fbaba91 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:06.575896 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2a84cd-7bc7-4fea-a5da-22c12fbaba91 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:06.581751 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f09a519-01ee-4b37-ab16-7480117ef345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:06.582807 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09a519-01ee-4b37-ab16-7480117ef345 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:06.583091 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09a519-01ee-4b37-ab16-7480117ef345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:06.583362 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09a519-01ee-4b37-ab16-7480117ef345 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:06.584209 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f09a519-01ee-4b37-ab16-7480117ef345 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:06.586232 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f09a519-01ee-4b37-ab16-7480117ef345 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:06.599898 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 969/1910] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:28:06 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:28:06.607175 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7556bf22-1683-41a3-843d-1a116a3b11ff admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:06.625671 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7556bf22-1683-41a3-843d-1a116a3b11ff admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:06.631880 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fa3e76-8a04-49db-b78a-8cea83d9307b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:06.632652 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa3e76-8a04-49db-b78a-8cea83d9307b admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:06.632955 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa3e76-8a04-49db-b78a-8cea83d9307b admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:06.633286 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa3e76-8a04-49db-b78a-8cea83d9307b admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:06.634625 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fa3e76-8a04-49db-b78a-8cea83d9307b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:06.636011 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fa3e76-8a04-49db-b78a-8cea83d9307b admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:06.652174 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 942/1911] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:28:06 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:28:06.653404 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f09a519-01ee-4b37-ab16-7480117ef345 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:06.667473 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f09a519-01ee-4b37-ab16-7480117ef345 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:06.675330 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f29f799-4e45-445e-97cb-63b56a09bad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:06.676256 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f29f799-4e45-445e-97cb-63b56a09bad8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:06.676676 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f29f799-4e45-445e-97cb-63b56a09bad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:06.677500 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f29f799-4e45-445e-97cb-63b56a09bad8 admin admin] PATH_INFO: `/v3/projects/2300ef2eb9b241c188a6ed30941285cb/users/be72bf92ec45457e988c41393ad06e8d/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:06.717502 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f29f799-4e45-445e-97cb-63b56a09bad8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2300ef2eb9b241c188a6ed30941285cb, user_id=be72bf92ec45457e988c41393ad06e8d, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:06.720623 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f29f799-4e45-445e-97cb-63b56a09bad8 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:06.774088 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 970/1912] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:28:06 2022] PUT /identity/v3/projects/2300ef2eb9b241c188a6ed30941285cb/users/be72bf92ec45457e988c41393ad06e8d/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:06.785136 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ed126-1ed3-472b-82f1-96240be01ae6 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:06.785435 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ed126-1ed3-472b-82f1-96240be01ae6 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:06.785714 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ed126-1ed3-472b-82f1-96240be01ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:06.815365 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-db5ed126-1ed3-472b-82f1-96240be01ae6 None None] MFA Rules not processed for user `be72bf92ec45457e988c41393ad06e8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:06.816771 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-db5ed126-1ed3-472b-82f1-96240be01ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:06.904436 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 943/1913] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:28:06.908882 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f29f799-4e45-445e-97cb-63b56a09bad8 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:06.974911 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:06.995982 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f29f799-4e45-445e-97cb-63b56a09bad8 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:07.001296 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75eaa12-aaae-4dc1-8f1c-75faf7835fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:07.001974 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75eaa12-aaae-4dc1-8f1c-75faf7835fc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:07.002264 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75eaa12-aaae-4dc1-8f1c-75faf7835fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:07.002544 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75eaa12-aaae-4dc1-8f1c-75faf7835fc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:07.003245 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75eaa12-aaae-4dc1-8f1c-75faf7835fc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:07.005265 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75eaa12-aaae-4dc1-8f1c-75faf7835fc9 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:07.106470 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 971/1914] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:28:06 2022] POST /identity/v3/projects => generated 381 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:28:07.115290 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5ed126-1ed3-472b-82f1-96240be01ae6 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:07.127162 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5ed126-1ed3-472b-82f1-96240be01ae6 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:07.189510 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a27504-feb9-444c-a37d-6e56a903bdf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:07.190222 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a27504-feb9-444c-a37d-6e56a903bdf8 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:07.190585 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a27504-feb9-444c-a37d-6e56a903bdf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:07.190873 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a27504-feb9-444c-a37d-6e56a903bdf8 admin admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:07.191514 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a27504-feb9-444c-a37d-6e56a903bdf8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:07.193505 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a27504-feb9-444c-a37d-6e56a903bdf8 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:07.200463 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:07.331078 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 944/1915] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:28:07 2022] POST /identity/v3/users => generated 297 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:28:07.332155 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75eaa12-aaae-4dc1-8f1c-75faf7835fc9 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:07.342900 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75eaa12-aaae-4dc1-8f1c-75faf7835fc9 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:07.347456 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55841956-0ffe-4e25-a9ea-1b38132d280c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:07.348116 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55841956-0ffe-4e25-a9ea-1b38132d280c admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:07.348417 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55841956-0ffe-4e25-a9ea-1b38132d280c admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:07.348701 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55841956-0ffe-4e25-a9ea-1b38132d280c admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:07.349429 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55841956-0ffe-4e25-a9ea-1b38132d280c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:07.351026 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55841956-0ffe-4e25-a9ea-1b38132d280c admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:07.368440 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 972/1916] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:28:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:28:07.369218 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a27504-feb9-444c-a37d-6e56a903bdf8 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:07.382861 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a27504-feb9-444c-a37d-6e56a903bdf8 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:07.385220 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfc4c3a-f984-43d8-9f1f-cf5515551577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:07.386111 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfc4c3a-f984-43d8-9f1f-cf5515551577 admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:07.386677 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfc4c3a-f984-43d8-9f1f-cf5515551577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:07.386677 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfc4c3a-f984-43d8-9f1f-cf5515551577 admin admin] PATH_INFO: `/v3/projects/780b2232eafa4e47b88a413727236adf/users/ee03c074231e4a9cb55a24ba86622d46/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:07.404574 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfc4c3a-f984-43d8-9f1f-cf5515551577 admin admin] RBAC: Authorizing `identity:create_grant(project_id=780b2232eafa4e47b88a413727236adf, user_id=ee03c074231e4a9cb55a24ba86622d46, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:07.407371 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfc4c3a-f984-43d8-9f1f-cf5515551577 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:07.510112 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 945/1917] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:28:07 2022] PUT /identity/v3/projects/780b2232eafa4e47b88a413727236adf/users/ee03c074231e4a9cb55a24ba86622d46/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:28:07.516802 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55841956-0ffe-4e25-a9ea-1b38132d280c admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:07.528489 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55841956-0ffe-4e25-a9ea-1b38132d280c admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:07.589074 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d1ea40-c3ce-4e22-b3eb-4481c7722c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:07.590963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d1ea40-c3ce-4e22-b3eb-4481c7722c2a admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:07.591318 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d1ea40-c3ce-4e22-b3eb-4481c7722c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:07.591600 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d1ea40-c3ce-4e22-b3eb-4481c7722c2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:07.592343 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d1ea40-c3ce-4e22-b3eb-4481c7722c2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:07.593955 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d1ea40-c3ce-4e22-b3eb-4481c7722c2a admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:07.604995 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 973/1918] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:28:07 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:28:07.609193 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfc4c3a-f984-43d8-9f1f-cf5515551577 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:07.619670 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfc4c3a-f984-43d8-9f1f-cf5515551577 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:07.624881 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d1d830-0d1b-4605-afb0-b7167a5d23d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:07.625821 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d1d830-0d1b-4605-afb0-b7167a5d23d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:07.626100 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d1d830-0d1b-4605-afb0-b7167a5d23d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:07.629533 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d1d830-0d1b-4605-afb0-b7167a5d23d5 admin admin] PATH_INFO: `/v3/projects/780b2232eafa4e47b88a413727236adf/users/ee03c074231e4a9cb55a24ba86622d46/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:07.629533 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d1d830-0d1b-4605-afb0-b7167a5d23d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=780b2232eafa4e47b88a413727236adf, user_id=ee03c074231e4a9cb55a24ba86622d46, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:07.631232 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d1d830-0d1b-4605-afb0-b7167a5d23d5 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:07.642548 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 946/1919] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:28:07 2022] PUT /identity/v3/projects/780b2232eafa4e47b88a413727236adf/users/ee03c074231e4a9cb55a24ba86622d46/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:07.654154 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0903f0-5b2c-4094-9911-47e3680e0847 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:07.656349 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0903f0-5b2c-4094-9911-47e3680e0847 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:07.656349 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0903f0-5b2c-4094-9911-47e3680e0847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:07.681367 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-4d0903f0-5b2c-4094-9911-47e3680e0847 None None] MFA Rules not processed for user `ee03c074231e4a9cb55a24ba86622d46`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:07.682578 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-4d0903f0-5b2c-4094-9911-47e3680e0847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:07.762515 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 974/1920] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:07 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:28:07.770966 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d1d830-0d1b-4605-afb0-b7167a5d23d5 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:07.772856 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-55d1d830-0d1b-4605-afb0-b7167a5d23d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:07.774495 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0903f0-5b2c-4094-9911-47e3680e0847 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:07.776714 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-4d0903f0-5b2c-4094-9911-47e3680e0847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:07.847264 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:07.849758 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d1d830-0d1b-4605-afb0-b7167a5d23d5 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:07.856038 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24603929-c794-481b-bc71-f4ade9fdc994 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:07.857095 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24603929-c794-481b-bc71-f4ade9fdc994 service cinder] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:07.857740 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24603929-c794-481b-bc71-f4ade9fdc994 service cinder] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:07.858215 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24603929-c794-481b-bc71-f4ade9fdc994 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:07.861338 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-24603929-c794-481b-bc71-f4ade9fdc994 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:07.875682 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0903f0-5b2c-4094-9911-47e3680e0847 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:07.884417 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edf3b70-f060-4458-89af-9b63c78ea7a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:07.884417 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf3b70-f060-4458-89af-9b63c78ea7a1 service cinder] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:07.884417 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf3b70-f060-4458-89af-9b63c78ea7a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:07.884417 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf3b70-f060-4458-89af-9b63c78ea7a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:07.895156 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edf3b70-f060-4458-89af-9b63c78ea7a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:07.896635 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edf3b70-f060-4458-89af-9b63c78ea7a1 service cinder] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:07.908779 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 975/1921] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:28:07 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:28:07.953940 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24603929-c794-481b-bc71-f4ade9fdc994 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:07.955384 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24603929-c794-481b-bc71-f4ade9fdc994 service cinder] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:07.968023 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 947/1922] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:28:07 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:28:08.485437 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edf3b70-f060-4458-89af-9b63c78ea7a1 service cinder] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:08.558385 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edf3b70-f060-4458-89af-9b63c78ea7a1 service cinder] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:08.563096 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da90417-9152-4793-a8be-3b251dd7dde6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:08.563971 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da90417-9152-4793-a8be-3b251dd7dde6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:08.564251 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da90417-9152-4793-a8be-3b251dd7dde6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:08.564710 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da90417-9152-4793-a8be-3b251dd7dde6 admin admin] PATH_INFO: `/v3/users/be72bf92ec45457e988c41393ad06e8d` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:08.565803 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da90417-9152-4793-a8be-3b251dd7dde6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be72bf92ec45457e988c41393ad06e8d)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:08.567704 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da90417-9152-4793-a8be-3b251dd7dde6 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:08.702499 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-1da90417-9152-4793-a8be-3b251dd7dde6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be72bf92ec45457e988c41393ad06e8d'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:08.716017 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-1da90417-9152-4793-a8be-3b251dd7dde6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be72bf92ec45457e988c41393ad06e8d'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:08.766378 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-1da90417-9152-4793-a8be-3b251dd7dde6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be72bf92ec45457e988c41393ad06e8d'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:08.767867 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-1da90417-9152-4793-a8be-3b251dd7dde6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be72bf92ec45457e988c41393ad06e8d'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:08.775733 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-1da90417-9152-4793-a8be-3b251dd7dde6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be72bf92ec45457e988c41393ad06e8d'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:08.782038 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 976/1923] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:28:08 2022] DELETE /identity/v3/users/be72bf92ec45457e988c41393ad06e8d => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:08.787191 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b8071-ddc7-4457-9850-470011a5ce7b None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:08.787191 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b8071-ddc7-4457-9850-470011a5ce7b None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:08.787191 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b8071-ddc7-4457-9850-470011a5ce7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:08.804847 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-a82b8071-ddc7-4457-9850-470011a5ce7b None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:08.806039 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-a82b8071-ddc7-4457-9850-470011a5ce7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:08.868415 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 948/1924] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:28:08.879186 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da90417-9152-4793-a8be-3b251dd7dde6 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:08.880856 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-1da90417-9152-4793-a8be-3b251dd7dde6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:08.940849 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da90417-9152-4793-a8be-3b251dd7dde6 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:08.944582 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14f5818-c8c6-477d-94dd-f47bf4c4390b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:08.945242 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f5818-c8c6-477d-94dd-f47bf4c4390b service neutron] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:08.945579 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f5818-c8c6-477d-94dd-f47bf4c4390b service neutron] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:08.945858 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f5818-c8c6-477d-94dd-f47bf4c4390b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:08.956493 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14f5818-c8c6-477d-94dd-f47bf4c4390b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:08.957928 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14f5818-c8c6-477d-94dd-f47bf4c4390b service neutron] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:08.969533 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 977/1925] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:28:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:28:09.551286 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82b8071-ddc7-4457-9850-470011a5ce7b None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:09.554467 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-a82b8071-ddc7-4457-9850-470011a5ce7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:09.565287 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:09.575340 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82b8071-ddc7-4457-9850-470011a5ce7b None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:09.579315 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac112a-81c6-4d55-903b-fa18bd078aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:09.581176 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac112a-81c6-4d55-903b-fa18bd078aba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:09.581581 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac112a-81c6-4d55-903b-fa18bd078aba admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:09.582095 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac112a-81c6-4d55-903b-fa18bd078aba admin admin] PATH_INFO: `/v3/projects/2300ef2eb9b241c188a6ed30941285cb` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:09.596867 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dac112a-81c6-4d55-903b-fa18bd078aba admin admin] RBAC: Authorizing `identity:delete_project(project_id=2300ef2eb9b241c188a6ed30941285cb)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:09.599215 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dac112a-81c6-4d55-903b-fa18bd078aba admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:09.747905 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-0dac112a-81c6-4d55-903b-fa18bd078aba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2300ef2eb9b241c188a6ed30941285cb', 'request_id': 'req-0dac112a-81c6-4d55-903b-fa18bd078aba'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:09.811967 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-0dac112a-81c6-4d55-903b-fa18bd078aba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2300ef2eb9b241c188a6ed30941285cb', 'request_id': 'req-0dac112a-81c6-4d55-903b-fa18bd078aba'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:09.820109 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-0dac112a-81c6-4d55-903b-fa18bd078aba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2300ef2eb9b241c188a6ed30941285cb', 'request_id': 'req-0dac112a-81c6-4d55-903b-fa18bd078aba'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:09.833718 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-0dac112a-81c6-4d55-903b-fa18bd078aba admin admin] The token cache is being invalidate because project 2300ef2eb9b241c188a6ed30941285cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:28:09.834010 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-0dac112a-81c6-4d55-903b-fa18bd078aba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:09.834693 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-0dac112a-81c6-4d55-903b-fa18bd078aba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:09.836407 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 949/1926] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:28:09 2022] DELETE /identity/v3/projects/2300ef2eb9b241c188a6ed30941285cb => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:09.841045 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14f5818-c8c6-477d-94dd-f47bf4c4390b service neutron] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:09.846264 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:09.851599 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14f5818-c8c6-477d-94dd-f47bf4c4390b service neutron] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:09.917596 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458d892c-5b4f-481d-9847-980edcd89914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:09.918741 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458d892c-5b4f-481d-9847-980edcd89914 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:09.919068 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458d892c-5b4f-481d-9847-980edcd89914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:09.919351 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458d892c-5b4f-481d-9847-980edcd89914 admin admin] PATH_INFO: `/v3/users/ee03c074231e4a9cb55a24ba86622d46` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:09.920785 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458d892c-5b4f-481d-9847-980edcd89914 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee03c074231e4a9cb55a24ba86622d46)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:09.922725 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458d892c-5b4f-481d-9847-980edcd89914 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:10.088961 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-458d892c-5b4f-481d-9847-980edcd89914 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee03c074231e4a9cb55a24ba86622d46'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:10.102043 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-458d892c-5b4f-481d-9847-980edcd89914 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee03c074231e4a9cb55a24ba86622d46'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:10.154890 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-458d892c-5b4f-481d-9847-980edcd89914 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee03c074231e4a9cb55a24ba86622d46'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:10.155837 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-458d892c-5b4f-481d-9847-980edcd89914 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee03c074231e4a9cb55a24ba86622d46'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:10.164695 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-458d892c-5b4f-481d-9847-980edcd89914 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee03c074231e4a9cb55a24ba86622d46'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:10.175508 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 978/1927] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:28:09 2022] DELETE /identity/v3/users/ee03c074231e4a9cb55a24ba86622d46 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:10.791589 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac112a-81c6-4d55-903b-fa18bd078aba admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:10.793519 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-0dac112a-81c6-4d55-903b-fa18bd078aba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:10.809036 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac112a-81c6-4d55-903b-fa18bd078aba admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:10.872901 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acc885d-d8b3-4198-b4a4-c7ae26ab7421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:10.873558 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc885d-d8b3-4198-b4a4-c7ae26ab7421 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:10.873558 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc885d-d8b3-4198-b4a4-c7ae26ab7421 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:10.873840 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc885d-d8b3-4198-b4a4-c7ae26ab7421 admin admin] PATH_INFO: `/v3/projects/780b2232eafa4e47b88a413727236adf` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:10.875289 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acc885d-d8b3-4198-b4a4-c7ae26ab7421 admin admin] RBAC: Authorizing `identity:delete_project(project_id=780b2232eafa4e47b88a413727236adf)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:10.877302 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acc885d-d8b3-4198-b4a4-c7ae26ab7421 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:10.979512 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-8acc885d-d8b3-4198-b4a4-c7ae26ab7421 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '780b2232eafa4e47b88a413727236adf', 'request_id': 'req-8acc885d-d8b3-4198-b4a4-c7ae26ab7421'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:11.047223 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-8acc885d-d8b3-4198-b4a4-c7ae26ab7421 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '780b2232eafa4e47b88a413727236adf', 'request_id': 'req-8acc885d-d8b3-4198-b4a4-c7ae26ab7421'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:11.053859 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-8acc885d-d8b3-4198-b4a4-c7ae26ab7421 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '780b2232eafa4e47b88a413727236adf', 'request_id': 'req-8acc885d-d8b3-4198-b4a4-c7ae26ab7421'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:11.068409 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-8acc885d-d8b3-4198-b4a4-c7ae26ab7421 admin admin] The token cache is being invalidate because project 780b2232eafa4e47b88a413727236adf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:28:11.068905 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-8acc885d-d8b3-4198-b4a4-c7ae26ab7421 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:11.069583 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-8acc885d-d8b3-4198-b4a4-c7ae26ab7421 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:11.071206 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 950/1928] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:28:10 2022] DELETE /identity/v3/projects/780b2232eafa4e47b88a413727236adf => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:11.076471 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ef7e0-1755-45c1-a12c-0bfa537bf7ba None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:11.077039 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ef7e0-1755-45c1-a12c-0bfa537bf7ba None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:11.077177 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ef7e0-1755-45c1-a12c-0bfa537bf7ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:11.097677 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-ae4ef7e0-1755-45c1-a12c-0bfa537bf7ba None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:11.098889 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-ae4ef7e0-1755-45c1-a12c-0bfa537bf7ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:11.162366 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 979/1929] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:28:11.171217 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439f33ab-480f-4193-9d6e-44a724ab9f5d None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:11.171507 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439f33ab-480f-4193-9d6e-44a724ab9f5d None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:11.171784 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439f33ab-480f-4193-9d6e-44a724ab9f5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:11.190317 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-439f33ab-480f-4193-9d6e-44a724ab9f5d None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:11.191568 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-439f33ab-480f-4193-9d6e-44a724ab9f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:11.199809 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 951/1930] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:28:11.204529 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4ef7e0-1755-45c1-a12c-0bfa537bf7ba None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:11.215318 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4ef7e0-1755-45c1-a12c-0bfa537bf7ba None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:11.220218 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647fb4aa-0ffd-46db-8b18-a341ec738a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:11.220870 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647fb4aa-0ffd-46db-8b18-a341ec738a40 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:11.221190 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647fb4aa-0ffd-46db-8b18-a341ec738a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:11.221471 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647fb4aa-0ffd-46db-8b18-a341ec738a40 admin admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:11.222228 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647fb4aa-0ffd-46db-8b18-a341ec738a40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:28:11.222537 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647fb4aa-0ffd-46db-8b18-a341ec738a40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:11.224248 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647fb4aa-0ffd-46db-8b18-a341ec738a40 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:11.237913 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 980/1931] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:28:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:28:11.242489 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439f33ab-480f-4193-9d6e-44a724ab9f5d None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:11.252430 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439f33ab-480f-4193-9d6e-44a724ab9f5d None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:11.257501 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b17c94-1370-42f9-b4bd-aa070c937ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:11.258189 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b17c94-1370-42f9-b4bd-aa070c937ebc admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:11.258473 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b17c94-1370-42f9-b4bd-aa070c937ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:11.258746 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b17c94-1370-42f9-b4bd-aa070c937ebc admin admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:11.259449 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b17c94-1370-42f9-b4bd-aa070c937ebc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:11.261859 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b17c94-1370-42f9-b4bd-aa070c937ebc admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:11.266477 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:11.348797 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 952/1932] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:28:11 2022] POST /identity/v3/projects => generated 379 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:28:11.350164 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647fb4aa-0ffd-46db-8b18-a341ec738a40 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:11.361091 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647fb4aa-0ffd-46db-8b18-a341ec738a40 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:11.366231 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:11.419614 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacdba0d-a48b-46d4-820e-bda2942caf74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:11.420532 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacdba0d-a48b-46d4-820e-bda2942caf74 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:11.420821 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacdba0d-a48b-46d4-820e-bda2942caf74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:11.421102 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacdba0d-a48b-46d4-820e-bda2942caf74 admin admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:11.421821 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacdba0d-a48b-46d4-820e-bda2942caf74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:11.423817 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacdba0d-a48b-46d4-820e-bda2942caf74 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:11.551037 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 981/1933] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:28:11 2022] POST /identity/v3/users => generated 290 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:28:11.556298 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b17c94-1370-42f9-b4bd-aa070c937ebc admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:11.566954 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b17c94-1370-42f9-b4bd-aa070c937ebc admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:11.571708 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feb0e1a-c9e4-4158-b442-5c1c05914b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:11.572487 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb0e1a-c9e4-4158-b442-5c1c05914b74 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:11.572764 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb0e1a-c9e4-4158-b442-5c1c05914b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:11.573097 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb0e1a-c9e4-4158-b442-5c1c05914b74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:11.574374 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feb0e1a-c9e4-4158-b442-5c1c05914b74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:11.576046 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feb0e1a-c9e4-4158-b442-5c1c05914b74 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:11.587149 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 953/1934] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:28:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:28:11.592137 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacdba0d-a48b-46d4-820e-bda2942caf74 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:11.602233 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacdba0d-a48b-46d4-820e-bda2942caf74 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:11.607613 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532c56e8-555d-41de-acfb-ddb6325c6689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:11.608368 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532c56e8-555d-41de-acfb-ddb6325c6689 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:11.608692 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532c56e8-555d-41de-acfb-ddb6325c6689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:11.609182 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532c56e8-555d-41de-acfb-ddb6325c6689 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:11.609694 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532c56e8-555d-41de-acfb-ddb6325c6689 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:11.613721 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532c56e8-555d-41de-acfb-ddb6325c6689 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:11.629088 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 982/1935] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:28:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:28:11.634104 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feb0e1a-c9e4-4158-b442-5c1c05914b74 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:11.645370 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feb0e1a-c9e4-4158-b442-5c1c05914b74 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:11.649997 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdfdc5e-2e8c-4eec-9483-8184d2102446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:11.651007 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdfdc5e-2e8c-4eec-9483-8184d2102446 admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:11.651296 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdfdc5e-2e8c-4eec-9483-8184d2102446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:11.651580 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdfdc5e-2e8c-4eec-9483-8184d2102446 admin admin] PATH_INFO: `/v3/projects/f2b3ab7a314e4c88be1b01931eef9877/users/46481fb46d5f44bd80289c5465f386fe/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:11.667848 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdfdc5e-2e8c-4eec-9483-8184d2102446 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2b3ab7a314e4c88be1b01931eef9877, user_id=46481fb46d5f44bd80289c5465f386fe, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:11.673919 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdfdc5e-2e8c-4eec-9483-8184d2102446 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:11.771670 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 954/1936] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:28:11 2022] PUT /identity/v3/projects/f2b3ab7a314e4c88be1b01931eef9877/users/46481fb46d5f44bd80289c5465f386fe/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:11.781145 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18a721-5b3c-4802-b2ea-757858c7cf78 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:11.781546 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18a721-5b3c-4802-b2ea-757858c7cf78 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:11.781903 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18a721-5b3c-4802-b2ea-757858c7cf78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:11.811586 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-1e18a721-5b3c-4802-b2ea-757858c7cf78 None None] MFA Rules not processed for user `46481fb46d5f44bd80289c5465f386fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:11.812972 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-1e18a721-5b3c-4802-b2ea-757858c7cf78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:11.894664 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 983/1937] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:11 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:28:11.899367 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdfdc5e-2e8c-4eec-9483-8184d2102446 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:11.961612 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdfdc5e-2e8c-4eec-9483-8184d2102446 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:11.967425 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea01763-d467-4630-86a3-5cce2ccefd56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:11.968089 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea01763-d467-4630-86a3-5cce2ccefd56 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:11.968372 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea01763-d467-4630-86a3-5cce2ccefd56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:11.968679 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea01763-d467-4630-86a3-5cce2ccefd56 admin admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:11.969384 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea01763-d467-4630-86a3-5cce2ccefd56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:11.971258 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea01763-d467-4630-86a3-5cce2ccefd56 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:12.083733 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 955/1938] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:28:11 2022] POST /identity/v3/projects => generated 381 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:28:12.087973 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e18a721-5b3c-4802-b2ea-757858c7cf78 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:12.100928 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e18a721-5b3c-4802-b2ea-757858c7cf78 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:12.166514 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7291b52f-7a86-4cf9-acbc-e46de760a014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:12.167176 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7291b52f-7a86-4cf9-acbc-e46de760a014 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:12.167466 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7291b52f-7a86-4cf9-acbc-e46de760a014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:12.167860 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7291b52f-7a86-4cf9-acbc-e46de760a014 admin admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:12.168774 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7291b52f-7a86-4cf9-acbc-e46de760a014 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:12.171068 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7291b52f-7a86-4cf9-acbc-e46de760a014 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:12.247891 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 984/1939] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:28:12 2022] POST /identity/v3/users => generated 297 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:28:12.252210 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea01763-d467-4630-86a3-5cce2ccefd56 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:12.264631 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea01763-d467-4630-86a3-5cce2ccefd56 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:12.270295 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd2ea74-7ca0-4007-8743-1ecb6b345d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:12.271213 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd2ea74-7ca0-4007-8743-1ecb6b345d6f admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:12.271487 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd2ea74-7ca0-4007-8743-1ecb6b345d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:12.271765 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd2ea74-7ca0-4007-8743-1ecb6b345d6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:12.272563 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd2ea74-7ca0-4007-8743-1ecb6b345d6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:12.274228 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd2ea74-7ca0-4007-8743-1ecb6b345d6f admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:12.287417 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 956/1940] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:28:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:28:12.291854 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7291b52f-7a86-4cf9-acbc-e46de760a014 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:12.303345 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7291b52f-7a86-4cf9-acbc-e46de760a014 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:12.311689 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03829fe0-ba8e-4fbc-83bc-4a678462dbf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:12.313268 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03829fe0-ba8e-4fbc-83bc-4a678462dbf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:12.313636 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03829fe0-ba8e-4fbc-83bc-4a678462dbf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:12.314022 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03829fe0-ba8e-4fbc-83bc-4a678462dbf5 admin admin] PATH_INFO: `/v3/projects/ca3ade0bc3734d9781317ce8d2f5f2d5/users/27c1740ca79946599b773a0511cdd194/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:12.316977 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:12.332692 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03829fe0-ba8e-4fbc-83bc-4a678462dbf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca3ade0bc3734d9781317ce8d2f5f2d5, user_id=27c1740ca79946599b773a0511cdd194, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:12.335319 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03829fe0-ba8e-4fbc-83bc-4a678462dbf5 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:12.390512 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 985/1941] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:28:12 2022] PUT /identity/v3/projects/ca3ade0bc3734d9781317ce8d2f5f2d5/users/27c1740ca79946599b773a0511cdd194/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:28:12.394760 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd2ea74-7ca0-4007-8743-1ecb6b345d6f admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:12.405826 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd2ea74-7ca0-4007-8743-1ecb6b345d6f admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:12.467508 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40174fd-d15f-4fba-92bd-16385b2266f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:12.468187 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40174fd-d15f-4fba-92bd-16385b2266f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:12.468530 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40174fd-d15f-4fba-92bd-16385b2266f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:12.468801 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40174fd-d15f-4fba-92bd-16385b2266f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:12.469531 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40174fd-d15f-4fba-92bd-16385b2266f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:12.471129 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40174fd-d15f-4fba-92bd-16385b2266f6 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:12.481467 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 957/1942] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:28:12 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:28:12.486056 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03829fe0-ba8e-4fbc-83bc-4a678462dbf5 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:12.497711 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03829fe0-ba8e-4fbc-83bc-4a678462dbf5 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:12.502661 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d68358-2fa6-4655-9afa-21df849d311e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:12.504806 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d68358-2fa6-4655-9afa-21df849d311e admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:12.504806 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d68358-2fa6-4655-9afa-21df849d311e admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:12.504806 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d68358-2fa6-4655-9afa-21df849d311e admin admin] PATH_INFO: `/v3/projects/ca3ade0bc3734d9781317ce8d2f5f2d5/users/27c1740ca79946599b773a0511cdd194/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:12.506213 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d68358-2fa6-4655-9afa-21df849d311e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca3ade0bc3734d9781317ce8d2f5f2d5, user_id=27c1740ca79946599b773a0511cdd194, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:12.508901 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d68358-2fa6-4655-9afa-21df849d311e admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:12.520456 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 986/1943] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:28:12 2022] PUT /identity/v3/projects/ca3ade0bc3734d9781317ce8d2f5f2d5/users/27c1740ca79946599b773a0511cdd194/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:12.531758 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb4d3a8-de89-459e-a233-19fd74eb6093 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:12.531962 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb4d3a8-de89-459e-a233-19fd74eb6093 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:12.532266 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb4d3a8-de89-459e-a233-19fd74eb6093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:12.561368 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-9bb4d3a8-de89-459e-a233-19fd74eb6093 None None] MFA Rules not processed for user `27c1740ca79946599b773a0511cdd194`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:12.562636 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-9bb4d3a8-de89-459e-a233-19fd74eb6093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:12.572173 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:12.633297 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 958/1944] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:12 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:28:12.645588 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d68358-2fa6-4655-9afa-21df849d311e admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:12.647459 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-61d68358-2fa6-4655-9afa-21df849d311e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:12.721808 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d68358-2fa6-4655-9afa-21df849d311e admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:12.725516 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b19c0cf-14b0-480f-9d5e-a59b34ac1fb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:12.726201 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b19c0cf-14b0-480f-9d5e-a59b34ac1fb5 service cinder] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:12.726485 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b19c0cf-14b0-480f-9d5e-a59b34ac1fb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:12.726760 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b19c0cf-14b0-480f-9d5e-a59b34ac1fb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:12.735874 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b19c0cf-14b0-480f-9d5e-a59b34ac1fb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:12.737326 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b19c0cf-14b0-480f-9d5e-a59b34ac1fb5 service cinder] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:12.747375 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 987/1945] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:28:12 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:28:12.972987 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb4d3a8-de89-459e-a233-19fd74eb6093 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:12.983033 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb4d3a8-de89-459e-a233-19fd74eb6093 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:12.987364 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca1e8b5-6e21-420c-ac7b-ffddadb7464b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:12.988011 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1e8b5-6e21-420c-ac7b-ffddadb7464b service cinder] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:12.988291 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1e8b5-6e21-420c-ac7b-ffddadb7464b service cinder] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:12.988592 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1e8b5-6e21-420c-ac7b-ffddadb7464b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:13.056491 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca1e8b5-6e21-420c-ac7b-ffddadb7464b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:13.057944 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca1e8b5-6e21-420c-ac7b-ffddadb7464b service cinder] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:13.068622 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 959/1946] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:28:12 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:28:13.604854 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b19c0cf-14b0-480f-9d5e-a59b34ac1fb5 service cinder] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:13.680499 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b19c0cf-14b0-480f-9d5e-a59b34ac1fb5 service cinder] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:13.686270 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb63e4ae-8b19-40bf-976a-c18f68a464a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:13.687223 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb63e4ae-8b19-40bf-976a-c18f68a464a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:13.693599 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb63e4ae-8b19-40bf-976a-c18f68a464a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:13.693599 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb63e4ae-8b19-40bf-976a-c18f68a464a3 admin admin] PATH_INFO: `/v3/users/46481fb46d5f44bd80289c5465f386fe` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:13.693599 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb63e4ae-8b19-40bf-976a-c18f68a464a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46481fb46d5f44bd80289c5465f386fe)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:13.693599 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb63e4ae-8b19-40bf-976a-c18f68a464a3 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:13.744668 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca1e8b5-6e21-420c-ac7b-ffddadb7464b service cinder] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:13.747538 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-4ca1e8b5-6e21-420c-ac7b-ffddadb7464b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:13.765019 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca1e8b5-6e21-420c-ac7b-ffddadb7464b service cinder] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:13.769792 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff12fa4-47fb-46fd-b711-227b1a5631c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:13.772632 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff12fa4-47fb-46fd-b711-227b1a5631c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:13.772930 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff12fa4-47fb-46fd-b711-227b1a5631c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:13.773215 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff12fa4-47fb-46fd-b711-227b1a5631c0 admin admin] PATH_INFO: `/v3/users/21e33a1cf7294b92b870e7e89968a9d4` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:13.774568 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff12fa4-47fb-46fd-b711-227b1a5631c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21e33a1cf7294b92b870e7e89968a9d4)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:13.779488 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff12fa4-47fb-46fd-b711-227b1a5631c0 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:13.925101 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-fb63e4ae-8b19-40bf-976a-c18f68a464a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46481fb46d5f44bd80289c5465f386fe'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:13.943779 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-fb63e4ae-8b19-40bf-976a-c18f68a464a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46481fb46d5f44bd80289c5465f386fe'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:14.016284 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-3ff12fa4-47fb-46fd-b711-227b1a5631c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21e33a1cf7294b92b870e7e89968a9d4'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:14.032511 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-3ff12fa4-47fb-46fd-b711-227b1a5631c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21e33a1cf7294b92b870e7e89968a9d4'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:14.033613 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-3ff12fa4-47fb-46fd-b711-227b1a5631c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21e33a1cf7294b92b870e7e89968a9d4'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:14.045284 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-3ff12fa4-47fb-46fd-b711-227b1a5631c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21e33a1cf7294b92b870e7e89968a9d4'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:14.046141 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-3ff12fa4-47fb-46fd-b711-227b1a5631c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21e33a1cf7294b92b870e7e89968a9d4'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:14.047288 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-fb63e4ae-8b19-40bf-976a-c18f68a464a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46481fb46d5f44bd80289c5465f386fe'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:14.048250 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-fb63e4ae-8b19-40bf-976a-c18f68a464a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46481fb46d5f44bd80289c5465f386fe'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:14.052312 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:14.062102 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-fb63e4ae-8b19-40bf-976a-c18f68a464a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46481fb46d5f44bd80289c5465f386fe'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:14.072147 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 988/1947] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:28:13 2022] DELETE /identity/v3/users/46481fb46d5f44bd80289c5465f386fe => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:14.078309 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfccfe0c-5133-4806-858b-79d317587844 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:14.078717 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfccfe0c-5133-4806-858b-79d317587844 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:14.079001 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfccfe0c-5133-4806-858b-79d317587844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:14.134052 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-bfccfe0c-5133-4806-858b-79d317587844 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:14.134052 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-bfccfe0c-5133-4806-858b-79d317587844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:14.136159 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 960/1948] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:28:13 2022] DELETE /identity/v3/users/21e33a1cf7294b92b870e7e89968a9d4 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:28:14.141473 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c99821-ebda-4cfe-8b6e-b91f61e52986 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:14.141847 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c99821-ebda-4cfe-8b6e-b91f61e52986 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:14.142354 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c99821-ebda-4cfe-8b6e-b91f61e52986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:14.164316 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-27c99821-ebda-4cfe-8b6e-b91f61e52986 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:14.166570 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-27c99821-ebda-4cfe-8b6e-b91f61e52986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:14.191488 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:14.271326 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 989/1949] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:28:14.309326 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfccfe0c-5133-4806-858b-79d317587844 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:14.311938 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-bfccfe0c-5133-4806-858b-79d317587844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:14.381319 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 961/1950] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:28:14.397176 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c99821-ebda-4cfe-8b6e-b91f61e52986 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:14.399867 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-27c99821-ebda-4cfe-8b6e-b91f61e52986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:14.422459 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfccfe0c-5133-4806-858b-79d317587844 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:14.522403 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492d3a88-25ae-4e97-b22d-a6270b7ed672 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:14.523245 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492d3a88-25ae-4e97-b22d-a6270b7ed672 service neutron] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:14.523539 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492d3a88-25ae-4e97-b22d-a6270b7ed672 service neutron] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:14.523894 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492d3a88-25ae-4e97-b22d-a6270b7ed672 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:14.536696 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492d3a88-25ae-4e97-b22d-a6270b7ed672 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:14.538402 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492d3a88-25ae-4e97-b22d-a6270b7ed672 service neutron] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:14.546725 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c99821-ebda-4cfe-8b6e-b91f61e52986 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:14.557251 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 990/1951] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:28:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:28:14.557426 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709bfbbd-7c13-42c3-912f-004538892612 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:14.557426 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709bfbbd-7c13-42c3-912f-004538892612 service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:14.557426 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709bfbbd-7c13-42c3-912f-004538892612 service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:14.557426 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709bfbbd-7c13-42c3-912f-004538892612 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:14.564152 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709bfbbd-7c13-42c3-912f-004538892612 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:14.565842 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709bfbbd-7c13-42c3-912f-004538892612 service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:14.606218 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 962/1952] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:28:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:28:15.378532 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492d3a88-25ae-4e97-b22d-a6270b7ed672 service neutron] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:15.381163 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-492d3a88-25ae-4e97-b22d-a6270b7ed672 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:15.400210 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492d3a88-25ae-4e97-b22d-a6270b7ed672 service neutron] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:15.409470 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226c82f5-0e3b-4bd1-a014-94d1610d02a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:15.410782 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c82f5-0e3b-4bd1-a014-94d1610d02a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:15.411079 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c82f5-0e3b-4bd1-a014-94d1610d02a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:15.411876 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c82f5-0e3b-4bd1-a014-94d1610d02a1 admin admin] PATH_INFO: `/v3/projects/f2b3ab7a314e4c88be1b01931eef9877` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:15.413284 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226c82f5-0e3b-4bd1-a014-94d1610d02a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2b3ab7a314e4c88be1b01931eef9877)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:15.415570 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226c82f5-0e3b-4bd1-a014-94d1610d02a1 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:15.649572 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709bfbbd-7c13-42c3-912f-004538892612 service neutron] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:15.652679 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-709bfbbd-7c13-42c3-912f-004538892612 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:15.689361 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709bfbbd-7c13-42c3-912f-004538892612 service neutron] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:15.697069 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:15.703095 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-226c82f5-0e3b-4bd1-a014-94d1610d02a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2b3ab7a314e4c88be1b01931eef9877', 'request_id': 'req-226c82f5-0e3b-4bd1-a014-94d1610d02a1'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:15.717818 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-226c82f5-0e3b-4bd1-a014-94d1610d02a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2b3ab7a314e4c88be1b01931eef9877', 'request_id': 'req-226c82f5-0e3b-4bd1-a014-94d1610d02a1'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:15.736930 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-226c82f5-0e3b-4bd1-a014-94d1610d02a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2b3ab7a314e4c88be1b01931eef9877', 'request_id': 'req-226c82f5-0e3b-4bd1-a014-94d1610d02a1'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:15.804308 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e9eff5-b644-48ab-9211-530509922f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:15.805230 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e9eff5-b644-48ab-9211-530509922f62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:15.805516 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e9eff5-b644-48ab-9211-530509922f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:15.805794 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e9eff5-b644-48ab-9211-530509922f62 admin admin] PATH_INFO: `/v3/projects/66c9c09c19a94a869e4df6d3c6d32c27` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:15.807035 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e9eff5-b644-48ab-9211-530509922f62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66c9c09c19a94a869e4df6d3c6d32c27)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:15.809826 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e9eff5-b644-48ab-9211-530509922f62 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:15.870034 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-226c82f5-0e3b-4bd1-a014-94d1610d02a1 admin admin] The token cache is being invalidate because project f2b3ab7a314e4c88be1b01931eef9877 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:28:15.870450 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-226c82f5-0e3b-4bd1-a014-94d1610d02a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:15.871983 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-226c82f5-0e3b-4bd1-a014-94d1610d02a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:15.873857 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 991/1953] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:28:15 2022] DELETE /identity/v3/projects/f2b3ab7a314e4c88be1b01931eef9877 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:15.878112 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226c82f5-0e3b-4bd1-a014-94d1610d02a1 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:15.879508 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-226c82f5-0e3b-4bd1-a014-94d1610d02a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:15.893771 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226c82f5-0e3b-4bd1-a014-94d1610d02a1 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:15.899011 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ff9262-728e-4872-b43c-a4a2c6ab0db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:15.899460 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ff9262-728e-4872-b43c-a4a2c6ab0db1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:15.899747 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ff9262-728e-4872-b43c-a4a2c6ab0db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:15.900028 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ff9262-728e-4872-b43c-a4a2c6ab0db1 admin admin] PATH_INFO: `/v3/users/27c1740ca79946599b773a0511cdd194` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:15.901412 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ff9262-728e-4872-b43c-a4a2c6ab0db1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27c1740ca79946599b773a0511cdd194)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:15.903381 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ff9262-728e-4872-b43c-a4a2c6ab0db1 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:16.059394 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-33e9eff5-b644-48ab-9211-530509922f62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66c9c09c19a94a869e4df6d3c6d32c27', 'request_id': 'req-33e9eff5-b644-48ab-9211-530509922f62'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:16.103271 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-56ff9262-728e-4872-b43c-a4a2c6ab0db1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27c1740ca79946599b773a0511cdd194'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:16.120973 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-33e9eff5-b644-48ab-9211-530509922f62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66c9c09c19a94a869e4df6d3c6d32c27', 'request_id': 'req-33e9eff5-b644-48ab-9211-530509922f62'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:16.122811 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-56ff9262-728e-4872-b43c-a4a2c6ab0db1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27c1740ca79946599b773a0511cdd194'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:16.131089 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:16.134346 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-33e9eff5-b644-48ab-9211-530509922f62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66c9c09c19a94a869e4df6d3c6d32c27', 'request_id': 'req-33e9eff5-b644-48ab-9211-530509922f62'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:16.165818 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-33e9eff5-b644-48ab-9211-530509922f62 admin admin] The token cache is being invalidate because project 66c9c09c19a94a869e4df6d3c6d32c27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:28:16.166116 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-33e9eff5-b644-48ab-9211-530509922f62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:16.169236 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-33e9eff5-b644-48ab-9211-530509922f62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:16.173393 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 963/1954] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:28:15 2022] DELETE /identity/v3/projects/66c9c09c19a94a869e4df6d3c6d32c27 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:28:16.181234 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482cc8b-38f9-4a8f-9d1d-146a6709676a None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:16.181893 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482cc8b-38f9-4a8f-9d1d-146a6709676a None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:16.182514 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482cc8b-38f9-4a8f-9d1d-146a6709676a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:16.206584 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-56ff9262-728e-4872-b43c-a4a2c6ab0db1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27c1740ca79946599b773a0511cdd194'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:16.206584 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-56ff9262-728e-4872-b43c-a4a2c6ab0db1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27c1740ca79946599b773a0511cdd194'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:16.223111 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-5482cc8b-38f9-4a8f-9d1d-146a6709676a None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:16.225324 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-5482cc8b-38f9-4a8f-9d1d-146a6709676a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:16.230834 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-56ff9262-728e-4872-b43c-a4a2c6ab0db1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27c1740ca79946599b773a0511cdd194'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:16.252302 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 992/1955] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:28:15 2022] DELETE /identity/v3/users/27c1740ca79946599b773a0511cdd194 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:16.357915 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 964/1956] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:28:16.372724 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08573d0e-91d7-4fbf-aa69-373be5170d23 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:16.375117 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08573d0e-91d7-4fbf-aa69-373be5170d23 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:16.376018 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08573d0e-91d7-4fbf-aa69-373be5170d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:16.411877 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-08573d0e-91d7-4fbf-aa69-373be5170d23 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:16.414633 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-08573d0e-91d7-4fbf-aa69-373be5170d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:16.508440 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 993/1957] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:28:16.513795 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5482cc8b-38f9-4a8f-9d1d-146a6709676a None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:16.565373 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5482cc8b-38f9-4a8f-9d1d-146a6709676a None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:16.581452 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fe45ce-00f3-4748-a1e6-e7464f35794f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:16.582436 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe45ce-00f3-4748-a1e6-e7464f35794f admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:16.582984 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe45ce-00f3-4748-a1e6-e7464f35794f admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:16.583265 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe45ce-00f3-4748-a1e6-e7464f35794f admin admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:16.584189 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fe45ce-00f3-4748-a1e6-e7464f35794f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:28:16.586049 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fe45ce-00f3-4748-a1e6-e7464f35794f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:16.588661 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fe45ce-00f3-4748-a1e6-e7464f35794f admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:16.608072 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 965/1958] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:28:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:28:16.614174 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08573d0e-91d7-4fbf-aa69-373be5170d23 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:16.626605 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08573d0e-91d7-4fbf-aa69-373be5170d23 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:16.641151 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b294e050-a1b1-4163-940e-25e6d62e47eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:16.642118 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b294e050-a1b1-4163-940e-25e6d62e47eb admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:16.642404 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b294e050-a1b1-4163-940e-25e6d62e47eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:16.643064 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b294e050-a1b1-4163-940e-25e6d62e47eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:16.643395 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b294e050-a1b1-4163-940e-25e6d62e47eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:16.647881 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b294e050-a1b1-4163-940e-25e6d62e47eb admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:16.786499 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 994/1959] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:28:16 2022] POST /identity/v3/projects => generated 371 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:28:16.790385 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fe45ce-00f3-4748-a1e6-e7464f35794f admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:16.814061 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fe45ce-00f3-4748-a1e6-e7464f35794f admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:16.912713 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b294e050-a1b1-4163-940e-25e6d62e47eb admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:16.915065 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-b294e050-a1b1-4163-940e-25e6d62e47eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:16.936906 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a98bcd9-c976-4551-9ef5-aaebd3958975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:16.937980 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a98bcd9-c976-4551-9ef5-aaebd3958975 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:16.938471 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a98bcd9-c976-4551-9ef5-aaebd3958975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:16.938852 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a98bcd9-c976-4551-9ef5-aaebd3958975 admin admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:16.940211 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a98bcd9-c976-4551-9ef5-aaebd3958975 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:16.945162 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a98bcd9-c976-4551-9ef5-aaebd3958975 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:16.985897 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:17.071965 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 966/1960] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:28:16 2022] POST /identity/v3/users => generated 286 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:28:17.077324 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a98bcd9-c976-4551-9ef5-aaebd3958975 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:17.080301 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-6a98bcd9-c976-4551-9ef5-aaebd3958975 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:17.082055 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b294e050-a1b1-4163-940e-25e6d62e47eb admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:17.090627 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fef7a9-19a5-46a4-b0f5-f064d09e2ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:17.091924 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fef7a9-19a5-46a4-b0f5-f064d09e2ea8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:17.092419 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fef7a9-19a5-46a4-b0f5-f064d09e2ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:17.092919 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fef7a9-19a5-46a4-b0f5-f064d09e2ea8 admin admin] PATH_INFO: `/v3/projects/ca3ade0bc3734d9781317ce8d2f5f2d5` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:17.095480 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fef7a9-19a5-46a4-b0f5-f064d09e2ea8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca3ade0bc3734d9781317ce8d2f5f2d5)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:17.102921 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fef7a9-19a5-46a4-b0f5-f064d09e2ea8 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:17.111193 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a98bcd9-c976-4551-9ef5-aaebd3958975 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:17.129869 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9930c43c-49e1-40d4-9fac-d0f123fd4c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:17.129869 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9930c43c-49e1-40d4-9fac-d0f123fd4c51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:17.129869 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9930c43c-49e1-40d4-9fac-d0f123fd4c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:17.129869 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9930c43c-49e1-40d4-9fac-d0f123fd4c51 admin admin] PATH_INFO: `/v3/users/47701eb554a24d049820beb63129e16f` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:17.131867 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9930c43c-49e1-40d4-9fac-d0f123fd4c51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47701eb554a24d049820beb63129e16f)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:17.135021 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9930c43c-49e1-40d4-9fac-d0f123fd4c51 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:17.391280 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-9930c43c-49e1-40d4-9fac-d0f123fd4c51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47701eb554a24d049820beb63129e16f'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:17.409601 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:17.411561 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-06fef7a9-19a5-46a4-b0f5-f064d09e2ea8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca3ade0bc3734d9781317ce8d2f5f2d5', 'request_id': 'req-06fef7a9-19a5-46a4-b0f5-f064d09e2ea8'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:17.414111 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-9930c43c-49e1-40d4-9fac-d0f123fd4c51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47701eb554a24d049820beb63129e16f'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:17.416378 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-9930c43c-49e1-40d4-9fac-d0f123fd4c51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47701eb554a24d049820beb63129e16f'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:17.426053 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-06fef7a9-19a5-46a4-b0f5-f064d09e2ea8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca3ade0bc3734d9781317ce8d2f5f2d5', 'request_id': 'req-06fef7a9-19a5-46a4-b0f5-f064d09e2ea8'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:17.435105 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-9930c43c-49e1-40d4-9fac-d0f123fd4c51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47701eb554a24d049820beb63129e16f'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:17.437379 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-9930c43c-49e1-40d4-9fac-d0f123fd4c51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47701eb554a24d049820beb63129e16f'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:17.438301 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-06fef7a9-19a5-46a4-b0f5-f064d09e2ea8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca3ade0bc3734d9781317ce8d2f5f2d5', 'request_id': 'req-06fef7a9-19a5-46a4-b0f5-f064d09e2ea8'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:17.531307 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 967/1961] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:28:17 2022] DELETE /identity/v3/users/47701eb554a24d049820beb63129e16f => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:17.532708 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9930c43c-49e1-40d4-9fac-d0f123fd4c51 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:17.534471 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-9930c43c-49e1-40d4-9fac-d0f123fd4c51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:17.591381 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-06fef7a9-19a5-46a4-b0f5-f064d09e2ea8 admin admin] The token cache is being invalidate because project ca3ade0bc3734d9781317ce8d2f5f2d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:28:17.591883 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-06fef7a9-19a5-46a4-b0f5-f064d09e2ea8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:17.593223 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-06fef7a9-19a5-46a4-b0f5-f064d09e2ea8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:17.596022 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 995/1962] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:28:16 2022] DELETE /identity/v3/projects/ca3ade0bc3734d9781317ce8d2f5f2d5 => generated 0 bytes in 684 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:28:17.600683 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1535c6-4757-4a3e-a9a3-3e9c9b7ee761 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:17.601017 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1535c6-4757-4a3e-a9a3-3e9c9b7ee761 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:17.601278 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1535c6-4757-4a3e-a9a3-3e9c9b7ee761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:17.633908 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9930c43c-49e1-40d4-9fac-d0f123fd4c51 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:17.636158 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f94da15-c793-42b0-87e5-e187b1a5ec1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:17.637189 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f94da15-c793-42b0-87e5-e187b1a5ec1b admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:17.637802 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f94da15-c793-42b0-87e5-e187b1a5ec1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:17.638237 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f94da15-c793-42b0-87e5-e187b1a5ec1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:17.639147 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f94da15-c793-42b0-87e5-e187b1a5ec1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:17.641473 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f94da15-c793-42b0-87e5-e187b1a5ec1b admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:17.647205 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-1e1535c6-4757-4a3e-a9a3-3e9c9b7ee761 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:17.648745 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-1e1535c6-4757-4a3e-a9a3-3e9c9b7ee761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:17.656333 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 968/1963] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:28:17.658031 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5be5ee-1e2c-4238-8286-fef6f2a10a37 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:17.658568 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5be5ee-1e2c-4238-8286-fef6f2a10a37 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:17.659275 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5be5ee-1e2c-4238-8286-fef6f2a10a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:17.676491 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 996/1964] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:28:17.681306 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1535c6-4757-4a3e-a9a3-3e9c9b7ee761 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:17.682711 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-1e1535c6-4757-4a3e-a9a3-3e9c9b7ee761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:17.692481 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-3e5be5ee-1e2c-4238-8286-fef6f2a10a37 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:17.694087 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-3e5be5ee-1e2c-4238-8286-fef6f2a10a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:17.706478 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 969/1965] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:28:17.707845 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5be5ee-1e2c-4238-8286-fef6f2a10a37 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:17.709540 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-3e5be5ee-1e2c-4238-8286-fef6f2a10a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:17.712939 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1535c6-4757-4a3e-a9a3-3e9c9b7ee761 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:17.728038 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcf8676-8a36-4634-8041-9bea58016d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:17.729705 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcf8676-8a36-4634-8041-9bea58016d0f admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:17.729705 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcf8676-8a36-4634-8041-9bea58016d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:17.731259 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcf8676-8a36-4634-8041-9bea58016d0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:17.732064 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcf8676-8a36-4634-8041-9bea58016d0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:17.733776 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcf8676-8a36-4634-8041-9bea58016d0f admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:17.769649 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 997/1966] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:28:17.773631 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ce660-d73b-4eca-acf7-584eb6ebd97b None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:17.774012 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ce660-d73b-4eca-acf7-584eb6ebd97b None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:17.774379 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ce660-d73b-4eca-acf7-584eb6ebd97b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:17.810699 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5be5ee-1e2c-4238-8286-fef6f2a10a37 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:17.816013 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5929f3-e4e4-429c-942d-aafe16ccf259 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:17.816838 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5929f3-e4e4-429c-942d-aafe16ccf259 service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:17.817710 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-b67ce660-d73b-4eca-acf7-584eb6ebd97b None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:17.817890 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5929f3-e4e4-429c-942d-aafe16ccf259 service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:17.818244 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5929f3-e4e4-429c-942d-aafe16ccf259 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:17.819236 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-b67ce660-d73b-4eca-acf7-584eb6ebd97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:17.830315 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5929f3-e4e4-429c-942d-aafe16ccf259 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:17.831248 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5929f3-e4e4-429c-942d-aafe16ccf259 service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:17.836936 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 998/1967] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:28:17.838211 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ce660-d73b-4eca-acf7-584eb6ebd97b None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:17.845068 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 970/1968] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:28:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:28:17.846212 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5929f3-e4e4-429c-942d-aafe16ccf259 service neutron] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:17.861507 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ce660-d73b-4eca-acf7-584eb6ebd97b None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:17.863058 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5929f3-e4e4-429c-942d-aafe16ccf259 service neutron] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:17.868953 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9980bdb-9f90-4e44-b97f-5dff28eaa357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:17.870224 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e9f792-2e98-4c58-8cf0-ec68fd0a27c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:17.870751 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9980bdb-9f90-4e44-b97f-5dff28eaa357 admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:17.871416 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9980bdb-9f90-4e44-b97f-5dff28eaa357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:17.871829 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9f792-2e98-4c58-8cf0-ec68fd0a27c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:17.872309 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9980bdb-9f90-4e44-b97f-5dff28eaa357 admin admin] PATH_INFO: `/v3/projects/444a0fac8c834c3cab5ba1df5a261aac/users/b6bbd45ee6964c4986f3c778ab8ad231/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:17.872521 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9f792-2e98-4c58-8cf0-ec68fd0a27c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:17.872812 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9f792-2e98-4c58-8cf0-ec68fd0a27c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:17.874872 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e9f792-2e98-4c58-8cf0-ec68fd0a27c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:28:17.875738 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e9f792-2e98-4c58-8cf0-ec68fd0a27c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:17.881225 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e9f792-2e98-4c58-8cf0-ec68fd0a27c4 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:17.900306 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 971/1969] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:28:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:28:17.907990 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9980bdb-9f90-4e44-b97f-5dff28eaa357 admin admin] RBAC: Authorizing `identity:create_grant(project_id=444a0fac8c834c3cab5ba1df5a261aac, user_id=b6bbd45ee6964c4986f3c778ab8ad231, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:17.908357 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9980bdb-9f90-4e44-b97f-5dff28eaa357 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:17.912361 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e9f792-2e98-4c58-8cf0-ec68fd0a27c4 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:17.925718 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e9f792-2e98-4c58-8cf0-ec68fd0a27c4 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:17.930996 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8818a9-3eda-43b3-b65d-1849672d8dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:17.931913 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8818a9-3eda-43b3-b65d-1849672d8dfc admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:17.932353 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8818a9-3eda-43b3-b65d-1849672d8dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:17.932830 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8818a9-3eda-43b3-b65d-1849672d8dfc admin admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:17.934240 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8818a9-3eda-43b3-b65d-1849672d8dfc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:17.936440 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8818a9-3eda-43b3-b65d-1849672d8dfc admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:17.975382 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 999/1970] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:28:17 2022] PUT /identity/v3/projects/444a0fac8c834c3cab5ba1df5a261aac/users/b6bbd45ee6964c4986f3c778ab8ad231/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:17.986000 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a083a7-7665-49f5-ae03-44efbd99e191 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:17.986713 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a083a7-7665-49f5-ae03-44efbd99e191 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:17.987174 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a083a7-7665-49f5-ae03-44efbd99e191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:18.009277 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-a9a083a7-7665-49f5-ae03-44efbd99e191 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:18.010545 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-a9a083a7-7665-49f5-ae03-44efbd99e191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:18.037457 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 972/1971] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:28:17 2022] POST /identity/v3/projects => generated 377 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:28:18.046509 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8818a9-3eda-43b3-b65d-1849672d8dfc admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:18.123060 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:18.151692 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8818a9-3eda-43b3-b65d-1849672d8dfc admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:18.154199 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1000/1972] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:28:18.159235 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608a2865-1790-4dc4-907a-e03785d19df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:18.159962 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a2865-1790-4dc4-907a-e03785d19df2 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:18.160252 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a2865-1790-4dc4-907a-e03785d19df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:18.160575 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a2865-1790-4dc4-907a-e03785d19df2 admin admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:18.161324 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608a2865-1790-4dc4-907a-e03785d19df2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:18.163581 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608a2865-1790-4dc4-907a-e03785d19df2 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:18.168120 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a083a7-7665-49f5-ae03-44efbd99e191 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:18.185533 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:18.238462 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a083a7-7665-49f5-ae03-44efbd99e191 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:18.309901 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d987969-0165-42c8-adb7-88fc3877bf7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:18.309901 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d987969-0165-42c8-adb7-88fc3877bf7a service neutron] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:18.309901 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d987969-0165-42c8-adb7-88fc3877bf7a service neutron] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:18.309901 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d987969-0165-42c8-adb7-88fc3877bf7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:18.319069 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d987969-0165-42c8-adb7-88fc3877bf7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:18.320504 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d987969-0165-42c8-adb7-88fc3877bf7a service neutron] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:18.337840 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1001/1973] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:28:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:28:18.551252 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 973/1974] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:28:18 2022] POST /identity/v3/users => generated 289 bytes in 508 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:28:18.556191 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d987969-0165-42c8-adb7-88fc3877bf7a service neutron] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:18.576821 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d987969-0165-42c8-adb7-88fc3877bf7a service neutron] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:18.585967 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020f786b-a1d6-4056-97f1-3e74410bc8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:18.587336 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f786b-a1d6-4056-97f1-3e74410bc8be admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:18.587931 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f786b-a1d6-4056-97f1-3e74410bc8be admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:18.588580 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f786b-a1d6-4056-97f1-3e74410bc8be admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:18.590406 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020f786b-a1d6-4056-97f1-3e74410bc8be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:18.593872 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020f786b-a1d6-4056-97f1-3e74410bc8be admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:18.618839 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1002/1975] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:28:18 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:28:18.622324 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608a2865-1790-4dc4-907a-e03785d19df2 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:18.637218 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608a2865-1790-4dc4-907a-e03785d19df2 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:18.644793 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd878df-2ee7-4c2a-bfca-9e19d2ac455c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:18.645517 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd878df-2ee7-4c2a-bfca-9e19d2ac455c admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:18.645799 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd878df-2ee7-4c2a-bfca-9e19d2ac455c admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:18.646082 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd878df-2ee7-4c2a-bfca-9e19d2ac455c admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:18.647533 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd878df-2ee7-4c2a-bfca-9e19d2ac455c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:18.649643 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd878df-2ee7-4c2a-bfca-9e19d2ac455c admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:18.669516 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 974/1976] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:28:18 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:28:18.673526 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020f786b-a1d6-4056-97f1-3e74410bc8be admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:18.695175 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020f786b-a1d6-4056-97f1-3e74410bc8be admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:18.705775 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4766a27-9bfa-4e24-8777-c6c5ec221735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:18.709817 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4766a27-9bfa-4e24-8777-c6c5ec221735 admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:18.709817 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4766a27-9bfa-4e24-8777-c6c5ec221735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:18.709817 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4766a27-9bfa-4e24-8777-c6c5ec221735 admin admin] PATH_INFO: `/v3/projects/778c91d98f0a4fbfa890057b0000f10a/users/da6b5526bcfc4d56a3a3f41eaadc9f7d/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:18.760933 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4766a27-9bfa-4e24-8777-c6c5ec221735 admin admin] RBAC: Authorizing `identity:create_grant(project_id=778c91d98f0a4fbfa890057b0000f10a, user_id=da6b5526bcfc4d56a3a3f41eaadc9f7d, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:18.769006 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4766a27-9bfa-4e24-8777-c6c5ec221735 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:18.851976 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1003/1977] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:28:18 2022] PUT /identity/v3/projects/778c91d98f0a4fbfa890057b0000f10a/users/da6b5526bcfc4d56a3a3f41eaadc9f7d/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:18.862349 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e6e7e-0d7c-4dfc-95ad-a25b4cb5d0a1 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:18.862952 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e6e7e-0d7c-4dfc-95ad-a25b4cb5d0a1 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:18.863117 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e6e7e-0d7c-4dfc-95ad-a25b4cb5d0a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:18.902571 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-b81e6e7e-0d7c-4dfc-95ad-a25b4cb5d0a1 None None] MFA Rules not processed for user `da6b5526bcfc4d56a3a3f41eaadc9f7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:18.903971 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-b81e6e7e-0d7c-4dfc-95ad-a25b4cb5d0a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:18.997332 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 975/1978] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:18 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:28:19.009159 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4766a27-9bfa-4e24-8777-c6c5ec221735 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:19.011022 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-a4766a27-9bfa-4e24-8777-c6c5ec221735 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:19.034986 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81e6e7e-0d7c-4dfc-95ad-a25b4cb5d0a1 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:19.036476 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-b81e6e7e-0d7c-4dfc-95ad-a25b4cb5d0a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:19.092689 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4766a27-9bfa-4e24-8777-c6c5ec221735 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:19.096993 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b3700e-ef55-4c9f-9b86-f25ed8431933 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:19.097672 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b3700e-ef55-4c9f-9b86-f25ed8431933 service cinder] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:19.098445 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b3700e-ef55-4c9f-9b86-f25ed8431933 service cinder] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:19.098445 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b3700e-ef55-4c9f-9b86-f25ed8431933 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:19.109549 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b3700e-ef55-4c9f-9b86-f25ed8431933 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:19.111452 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b3700e-ef55-4c9f-9b86-f25ed8431933 service cinder] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:19.122928 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1004/1979] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:28:19 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:28:19.149934 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81e6e7e-0d7c-4dfc-95ad-a25b4cb5d0a1 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:19.156475 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe84ad9-f255-4e23-8957-5cff4713299d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:19.157617 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe84ad9-f255-4e23-8957-5cff4713299d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:19.158026 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe84ad9-f255-4e23-8957-5cff4713299d admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:19.158414 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe84ad9-f255-4e23-8957-5cff4713299d admin admin] PATH_INFO: `/v3/projects/e45e6f9e8c344d7b9a21d92e27dafd9b` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:19.160359 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe84ad9-f255-4e23-8957-5cff4713299d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e45e6f9e8c344d7b9a21d92e27dafd9b)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:19.162982 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe84ad9-f255-4e23-8957-5cff4713299d admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:19.268076 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:19.270452 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-8fe84ad9-f255-4e23-8957-5cff4713299d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e45e6f9e8c344d7b9a21d92e27dafd9b', 'request_id': 'req-8fe84ad9-f255-4e23-8957-5cff4713299d'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:19.349717 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-8fe84ad9-f255-4e23-8957-5cff4713299d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e45e6f9e8c344d7b9a21d92e27dafd9b', 'request_id': 'req-8fe84ad9-f255-4e23-8957-5cff4713299d'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:19.361394 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-8fe84ad9-f255-4e23-8957-5cff4713299d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e45e6f9e8c344d7b9a21d92e27dafd9b', 'request_id': 'req-8fe84ad9-f255-4e23-8957-5cff4713299d'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:19.383570 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-8fe84ad9-f255-4e23-8957-5cff4713299d admin admin] The token cache is being invalidate because project e45e6f9e8c344d7b9a21d92e27dafd9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:28:19.383878 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-8fe84ad9-f255-4e23-8957-5cff4713299d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:19.384640 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-8fe84ad9-f255-4e23-8957-5cff4713299d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:19.386477 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 976/1980] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:28:19 2022] DELETE /identity/v3/projects/e45e6f9e8c344d7b9a21d92e27dafd9b => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:19.390731 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b3700e-ef55-4c9f-9b86-f25ed8431933 service cinder] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:19.405031 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b3700e-ef55-4c9f-9b86-f25ed8431933 service cinder] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:19.452161 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:19.495580 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cd981e-985b-4a4a-b115-dfd6aff8bdd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:19.497140 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd981e-985b-4a4a-b115-dfd6aff8bdd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:19.497437 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd981e-985b-4a4a-b115-dfd6aff8bdd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:19.497921 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd981e-985b-4a4a-b115-dfd6aff8bdd7 admin admin] PATH_INFO: `/v3/users/a97bab8260354bedb55c4b9e6c3cb57c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:19.500539 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cd981e-985b-4a4a-b115-dfd6aff8bdd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a97bab8260354bedb55c4b9e6c3cb57c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:19.502823 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cd981e-985b-4a4a-b115-dfd6aff8bdd7 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:19.809175 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-f7cd981e-985b-4a4a-b115-dfd6aff8bdd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a97bab8260354bedb55c4b9e6c3cb57c'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:19.835288 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-f7cd981e-985b-4a4a-b115-dfd6aff8bdd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a97bab8260354bedb55c4b9e6c3cb57c'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:19.976283 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-f7cd981e-985b-4a4a-b115-dfd6aff8bdd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a97bab8260354bedb55c4b9e6c3cb57c'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:19.976283 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-f7cd981e-985b-4a4a-b115-dfd6aff8bdd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a97bab8260354bedb55c4b9e6c3cb57c'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:20.018505 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-f7cd981e-985b-4a4a-b115-dfd6aff8bdd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a97bab8260354bedb55c4b9e6c3cb57c'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:20.055691 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1005/1981] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:28:19 2022] DELETE /identity/v3/users/a97bab8260354bedb55c4b9e6c3cb57c => generated 0 bytes in 661 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:21.040429 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe84ad9-f255-4e23-8957-5cff4713299d admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:21.040429 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-8fe84ad9-f255-4e23-8957-5cff4713299d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:21.059132 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe84ad9-f255-4e23-8957-5cff4713299d admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:21.153962 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ff7761-cad7-4169-9680-6151a339a4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:21.155508 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ff7761-cad7-4169-9680-6151a339a4c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:21.155871 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ff7761-cad7-4169-9680-6151a339a4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:21.156248 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ff7761-cad7-4169-9680-6151a339a4c5 admin admin] PATH_INFO: `/v3/projects/41a79443bf5a4ebb87d013ee6c543d20` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:21.158658 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ff7761-cad7-4169-9680-6151a339a4c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41a79443bf5a4ebb87d013ee6c543d20)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:21.161471 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ff7761-cad7-4169-9680-6151a339a4c5 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:21.292019 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-09ff7761-cad7-4169-9680-6151a339a4c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41a79443bf5a4ebb87d013ee6c543d20', 'request_id': 'req-09ff7761-cad7-4169-9680-6151a339a4c5'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:21.307412 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e172f7-9762-469e-8e0a-174751585baa None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:21.312745 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e172f7-9762-469e-8e0a-174751585baa None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:21.312745 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e172f7-9762-469e-8e0a-174751585baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:21.345288 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-e2e172f7-9762-469e-8e0a-174751585baa None None] MFA Rules not processed for user `b6bbd45ee6964c4986f3c778ab8ad231`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:21.346576 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-e2e172f7-9762-469e-8e0a-174751585baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:21.389272 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-09ff7761-cad7-4169-9680-6151a339a4c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41a79443bf5a4ebb87d013ee6c543d20', 'request_id': 'req-09ff7761-cad7-4169-9680-6151a339a4c5'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:21.398991 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-09ff7761-cad7-4169-9680-6151a339a4c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41a79443bf5a4ebb87d013ee6c543d20', 'request_id': 'req-09ff7761-cad7-4169-9680-6151a339a4c5'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:21.422463 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-09ff7761-cad7-4169-9680-6151a339a4c5 admin admin] The token cache is being invalidate because project 41a79443bf5a4ebb87d013ee6c543d20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:28:21.422956 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-09ff7761-cad7-4169-9680-6151a339a4c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:21.425270 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-09ff7761-cad7-4169-9680-6151a339a4c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:21.432575 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 977/1982] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:28:21 2022] DELETE /identity/v3/projects/41a79443bf5a4ebb87d013ee6c543d20 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:28:21.439262 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ee2d96-df8a-4999-b28b-a6c3b8a19127 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:21.441241 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ee2d96-df8a-4999-b28b-a6c3b8a19127 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:21.442191 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ee2d96-df8a-4999-b28b-a6c3b8a19127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:21.452122 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1006/1983] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:28:21.461459 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e172f7-9762-469e-8e0a-174751585baa None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:21.464285 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-e2e172f7-9762-469e-8e0a-174751585baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:21.468526 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-34ee2d96-df8a-4999-b28b-a6c3b8a19127 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:21.469807 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-34ee2d96-df8a-4999-b28b-a6c3b8a19127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:21.535205 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:21.550901 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e172f7-9762-469e-8e0a-174751585baa None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:21.554921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 978/1984] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:28:21.555063 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1856d5-7d01-4a05-a976-8b0f71c2b30d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:21.555795 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1856d5-7d01-4a05-a976-8b0f71c2b30d service cinder] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:21.556143 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1856d5-7d01-4a05-a976-8b0f71c2b30d service cinder] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:21.556520 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1856d5-7d01-4a05-a976-8b0f71c2b30d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:21.559020 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-4a1856d5-7d01-4a05-a976-8b0f71c2b30d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:21.563422 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d3c88-95f1-454d-800f-f611f9c607c0 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:21.563960 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d3c88-95f1-454d-800f-f611f9c607c0 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:21.564485 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d3c88-95f1-454d-800f-f611f9c607c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:21.571423 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1856d5-7d01-4a05-a976-8b0f71c2b30d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:21.573041 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1856d5-7d01-4a05-a976-8b0f71c2b30d service cinder] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:21.583684 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:21.584022 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1007/1985] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:28:21 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:28:21.590501 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-2d4d3c88-95f1-454d-800f-f611f9c607c0 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:21.592612 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-2d4d3c88-95f1-454d-800f-f611f9c607c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:21.600268 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 979/1986] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:28:21.604801 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1856d5-7d01-4a05-a976-8b0f71c2b30d service cinder] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:21.617415 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1856d5-7d01-4a05-a976-8b0f71c2b30d service cinder] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:21.622836 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82800763-23c1-4814-a1ed-cc4a926aa7fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:21.623550 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82800763-23c1-4814-a1ed-cc4a926aa7fc admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:21.623845 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82800763-23c1-4814-a1ed-cc4a926aa7fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:21.624426 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82800763-23c1-4814-a1ed-cc4a926aa7fc admin admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:21.625157 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82800763-23c1-4814-a1ed-cc4a926aa7fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:28:21.625724 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82800763-23c1-4814-a1ed-cc4a926aa7fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:21.627337 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82800763-23c1-4814-a1ed-cc4a926aa7fc admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:21.644289 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1008/1987] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:28:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:28:21.645173 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4d3c88-95f1-454d-800f-f611f9c607c0 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:21.655547 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4d3c88-95f1-454d-800f-f611f9c607c0 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:21.660453 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a79f248-cab8-48be-b3ba-5edbc1a0ce75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:21.661134 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a79f248-cab8-48be-b3ba-5edbc1a0ce75 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:21.661420 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a79f248-cab8-48be-b3ba-5edbc1a0ce75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:21.661702 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a79f248-cab8-48be-b3ba-5edbc1a0ce75 admin admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:21.662439 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a79f248-cab8-48be-b3ba-5edbc1a0ce75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:21.664524 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a79f248-cab8-48be-b3ba-5edbc1a0ce75 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:21.807739 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 980/1988] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:28:21 2022] POST /identity/v3/projects => generated 385 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:28:21.812845 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82800763-23c1-4814-a1ed-cc4a926aa7fc admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:21.825309 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82800763-23c1-4814-a1ed-cc4a926aa7fc admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:21.961377 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5459402-4659-4287-bd01-7a71d6ea50e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:21.962212 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5459402-4659-4287-bd01-7a71d6ea50e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:21.962310 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5459402-4659-4287-bd01-7a71d6ea50e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:21.962599 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5459402-4659-4287-bd01-7a71d6ea50e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:21.963956 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5459402-4659-4287-bd01-7a71d6ea50e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:21.967375 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5459402-4659-4287-bd01-7a71d6ea50e5 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:22.088453 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1009/1989] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:28:21 2022] POST /identity/v3/users => generated 293 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:28:22.092204 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a79f248-cab8-48be-b3ba-5edbc1a0ce75 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:22.103113 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a79f248-cab8-48be-b3ba-5edbc1a0ce75 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:22.108192 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1add0b-3934-4d94-9533-4fe90f0f6692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:22.108913 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1add0b-3934-4d94-9533-4fe90f0f6692 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:22.109204 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1add0b-3934-4d94-9533-4fe90f0f6692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:22.109484 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1add0b-3934-4d94-9533-4fe90f0f6692 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:22.110215 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1add0b-3934-4d94-9533-4fe90f0f6692 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:22.111842 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1add0b-3934-4d94-9533-4fe90f0f6692 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:22.129499 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 981/1990] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:28:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:28:22.132243 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5459402-4659-4287-bd01-7a71d6ea50e5 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:22.143351 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5459402-4659-4287-bd01-7a71d6ea50e5 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:22.149122 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236a6a3a-a7f4-47d3-9a10-7bc2f1d51e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:22.149742 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a6a3a-a7f4-47d3-9a10-7bc2f1d51e9d admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:22.150031 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a6a3a-a7f4-47d3-9a10-7bc2f1d51e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:22.150311 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a6a3a-a7f4-47d3-9a10-7bc2f1d51e9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:22.151066 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236a6a3a-a7f4-47d3-9a10-7bc2f1d51e9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:22.153504 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236a6a3a-a7f4-47d3-9a10-7bc2f1d51e9d admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:22.164972 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1010/1991] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:28:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:28:22.169339 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1add0b-3934-4d94-9533-4fe90f0f6692 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:22.179673 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1add0b-3934-4d94-9533-4fe90f0f6692 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:22.186857 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a418afb8-ca4f-45ee-84c0-5870ab503dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:22.186857 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a418afb8-ca4f-45ee-84c0-5870ab503dce admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:22.186857 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a418afb8-ca4f-45ee-84c0-5870ab503dce admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:22.186857 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a418afb8-ca4f-45ee-84c0-5870ab503dce admin admin] PATH_INFO: `/v3/projects/692d82e02ad043f5a38de17f754a8136/users/02dd7ab36bdf418582404649e7b46137/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:22.203468 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a418afb8-ca4f-45ee-84c0-5870ab503dce admin admin] RBAC: Authorizing `identity:create_grant(project_id=692d82e02ad043f5a38de17f754a8136, user_id=02dd7ab36bdf418582404649e7b46137, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:22.206220 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a418afb8-ca4f-45ee-84c0-5870ab503dce admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:22.315730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 982/1992] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:28:22 2022] PUT /identity/v3/projects/692d82e02ad043f5a38de17f754a8136/users/02dd7ab36bdf418582404649e7b46137/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:22.327941 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1fbffe-7c4d-4a8e-978c-109ccbfcca62 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:22.328548 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1fbffe-7c4d-4a8e-978c-109ccbfcca62 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:22.328548 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1fbffe-7c4d-4a8e-978c-109ccbfcca62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:22.359813 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-af1fbffe-7c4d-4a8e-978c-109ccbfcca62 None None] MFA Rules not processed for user `02dd7ab36bdf418582404649e7b46137`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:22.361117 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-af1fbffe-7c4d-4a8e-978c-109ccbfcca62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:22.453305 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1011/1993] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:28:22.463996 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a418afb8-ca4f-45ee-84c0-5870ab503dce admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:22.550751 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a418afb8-ca4f-45ee-84c0-5870ab503dce admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:22.555624 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e064fd-5d39-4680-9214-e500a8bfbb90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:22.557635 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e064fd-5d39-4680-9214-e500a8bfbb90 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:22.557635 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e064fd-5d39-4680-9214-e500a8bfbb90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:22.557635 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e064fd-5d39-4680-9214-e500a8bfbb90 admin admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:22.557635 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e064fd-5d39-4680-9214-e500a8bfbb90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:22.560196 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e064fd-5d39-4680-9214-e500a8bfbb90 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:22.652932 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1fbffe-7c4d-4a8e-978c-109ccbfcca62 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:22.654552 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 983/1994] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:28:22 2022] POST /identity/v3/projects => generated 383 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:28:22.666248 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1fbffe-7c4d-4a8e-978c-109ccbfcca62 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:22.707680 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:22.725654 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b95370d-a172-46b8-babe-b45857a1920b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:22.726326 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b95370d-a172-46b8-babe-b45857a1920b admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:22.726607 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b95370d-a172-46b8-babe-b45857a1920b admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:22.726880 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b95370d-a172-46b8-babe-b45857a1920b admin admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:22.727579 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b95370d-a172-46b8-babe-b45857a1920b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:22.729531 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b95370d-a172-46b8-babe-b45857a1920b admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:22.800958 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1012/1995] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:28:22 2022] POST /identity/v3/users => generated 298 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:28:22.805353 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e064fd-5d39-4680-9214-e500a8bfbb90 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:22.817135 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e064fd-5d39-4680-9214-e500a8bfbb90 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:22.821841 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f040c2-ca21-4b9e-96d8-c3c9a705110d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:22.829501 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f040c2-ca21-4b9e-96d8-c3c9a705110d admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:22.829501 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f040c2-ca21-4b9e-96d8-c3c9a705110d admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:22.829501 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f040c2-ca21-4b9e-96d8-c3c9a705110d admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:22.829501 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f040c2-ca21-4b9e-96d8-c3c9a705110d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:22.829501 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f040c2-ca21-4b9e-96d8-c3c9a705110d admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:22.836308 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 984/1996] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:28:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:28:22.840874 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b95370d-a172-46b8-babe-b45857a1920b admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:22.851264 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b95370d-a172-46b8-babe-b45857a1920b admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:22.857132 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72620817-3873-43f3-a0df-97b379952db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:22.858085 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72620817-3873-43f3-a0df-97b379952db8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:22.858367 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72620817-3873-43f3-a0df-97b379952db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:22.858643 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72620817-3873-43f3-a0df-97b379952db8 admin admin] PATH_INFO: `/v3/projects/9da2a8ec241e49b6bfe1143358d22b85/users/15d00d219f014f3c80d1f9d3331470ec/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:22.875778 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72620817-3873-43f3-a0df-97b379952db8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9da2a8ec241e49b6bfe1143358d22b85, user_id=15d00d219f014f3c80d1f9d3331470ec, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:22.878522 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72620817-3873-43f3-a0df-97b379952db8 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:22.943791 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1013/1997] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:28:22 2022] PUT /identity/v3/projects/9da2a8ec241e49b6bfe1143358d22b85/users/15d00d219f014f3c80d1f9d3331470ec/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:22.948010 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f040c2-ca21-4b9e-96d8-c3c9a705110d admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:22.958797 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f040c2-ca21-4b9e-96d8-c3c9a705110d admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:23.046014 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79001948-2b9a-48ba-aded-6dec2a4f3e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:23.046771 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79001948-2b9a-48ba-aded-6dec2a4f3e76 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:23.046990 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79001948-2b9a-48ba-aded-6dec2a4f3e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:23.047272 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79001948-2b9a-48ba-aded-6dec2a4f3e76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:23.048037 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79001948-2b9a-48ba-aded-6dec2a4f3e76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:23.049755 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79001948-2b9a-48ba-aded-6dec2a4f3e76 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:23.071898 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 985/1998] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:28:22 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:28:23.079439 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72620817-3873-43f3-a0df-97b379952db8 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:23.096884 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72620817-3873-43f3-a0df-97b379952db8 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:23.104221 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb0b43d-a839-4011-a438-cde6da38e7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:23.106016 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb0b43d-a839-4011-a438-cde6da38e7b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:23.106349 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb0b43d-a839-4011-a438-cde6da38e7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:23.106721 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb0b43d-a839-4011-a438-cde6da38e7b4 admin admin] PATH_INFO: `/v3/projects/9da2a8ec241e49b6bfe1143358d22b85/users/15d00d219f014f3c80d1f9d3331470ec/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:23.109207 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb0b43d-a839-4011-a438-cde6da38e7b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9da2a8ec241e49b6bfe1143358d22b85, user_id=15d00d219f014f3c80d1f9d3331470ec, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:23.112167 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb0b43d-a839-4011-a438-cde6da38e7b4 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:23.126534 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1014/1999] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:28:23 2022] PUT /identity/v3/projects/9da2a8ec241e49b6bfe1143358d22b85/users/15d00d219f014f3c80d1f9d3331470ec/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:23.137034 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe35c70-3f1d-4671-8dac-87045361c65d None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:23.137533 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe35c70-3f1d-4671-8dac-87045361c65d None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:23.137731 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe35c70-3f1d-4671-8dac-87045361c65d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:23.171314 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:23.174653 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-0fe35c70-3f1d-4671-8dac-87045361c65d None None] MFA Rules not processed for user `15d00d219f014f3c80d1f9d3331470ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:23.175998 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-0fe35c70-3f1d-4671-8dac-87045361c65d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:23.250771 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 986/2000] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:23 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:28:23.262810 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb0b43d-a839-4011-a438-cde6da38e7b4 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:23.264809 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-feb0b43d-a839-4011-a438-cde6da38e7b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:23.328964 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb0b43d-a839-4011-a438-cde6da38e7b4 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:23.332933 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbc9821-9de3-4d57-b86a-7d2bbfe38179 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:23.333596 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbc9821-9de3-4d57-b86a-7d2bbfe38179 service cinder] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:23.333882 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbc9821-9de3-4d57-b86a-7d2bbfe38179 service cinder] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:23.334160 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbc9821-9de3-4d57-b86a-7d2bbfe38179 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:23.344095 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbc9821-9de3-4d57-b86a-7d2bbfe38179 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:23.345552 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbc9821-9de3-4d57-b86a-7d2bbfe38179 service cinder] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:23.355263 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1015/2001] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:28:23 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:28:23.382361 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe35c70-3f1d-4671-8dac-87045361c65d None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:23.445128 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe35c70-3f1d-4671-8dac-87045361c65d None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:23.450382 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7ac568-3668-47ae-8db4-a5fd146c6043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:23.451261 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7ac568-3668-47ae-8db4-a5fd146c6043 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:23.451546 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7ac568-3668-47ae-8db4-a5fd146c6043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:23.451827 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7ac568-3668-47ae-8db4-a5fd146c6043 admin admin] PATH_INFO: `/v3/users/02dd7ab36bdf418582404649e7b46137` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:23.453059 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7ac568-3668-47ae-8db4-a5fd146c6043 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02dd7ab36bdf418582404649e7b46137)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:23.454972 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7ac568-3668-47ae-8db4-a5fd146c6043 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:23.668474 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-4a7ac568-3668-47ae-8db4-a5fd146c6043 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02dd7ab36bdf418582404649e7b46137'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:23.681166 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-4a7ac568-3668-47ae-8db4-a5fd146c6043 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02dd7ab36bdf418582404649e7b46137'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:23.681889 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-4a7ac568-3668-47ae-8db4-a5fd146c6043 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02dd7ab36bdf418582404649e7b46137'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:23.690099 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-4a7ac568-3668-47ae-8db4-a5fd146c6043 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02dd7ab36bdf418582404649e7b46137'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:23.690841 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-4a7ac568-3668-47ae-8db4-a5fd146c6043 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02dd7ab36bdf418582404649e7b46137'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:23.767898 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 987/2002] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:28:23 2022] DELETE /identity/v3/users/02dd7ab36bdf418582404649e7b46137 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:23.772420 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e80643-680b-4435-b53a-87bb063237fc None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:23.772799 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e80643-680b-4435-b53a-87bb063237fc None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:23.773080 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e80643-680b-4435-b53a-87bb063237fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:23.791805 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-05e80643-680b-4435-b53a-87bb063237fc None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:23.793294 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-05e80643-680b-4435-b53a-87bb063237fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:23.854783 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1016/2003] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:28:23.864758 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7ac568-3668-47ae-8db4-a5fd146c6043 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:23.866291 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-4a7ac568-3668-47ae-8db4-a5fd146c6043 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:23.890594 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:23.930031 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7ac568-3668-47ae-8db4-a5fd146c6043 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:23.934430 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70835009-4ef9-4b2a-992a-84f45c4f4177 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:23.935097 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70835009-4ef9-4b2a-992a-84f45c4f4177 service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:23.935387 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70835009-4ef9-4b2a-992a-84f45c4f4177 service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:23.935796 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70835009-4ef9-4b2a-992a-84f45c4f4177 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:23.945676 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70835009-4ef9-4b2a-992a-84f45c4f4177 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:23.947117 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70835009-4ef9-4b2a-992a-84f45c4f4177 service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:23.958753 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 988/2004] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:28:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:28:24.408191 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e80643-680b-4435-b53a-87bb063237fc None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:24.409745 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-05e80643-680b-4435-b53a-87bb063237fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:24.423652 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e80643-680b-4435-b53a-87bb063237fc None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:24.428598 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4564e677-5d99-4cdd-bd79-bbb23f126109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:24.429482 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4564e677-5d99-4cdd-bd79-bbb23f126109 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:24.429775 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4564e677-5d99-4cdd-bd79-bbb23f126109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:24.430054 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4564e677-5d99-4cdd-bd79-bbb23f126109 admin admin] PATH_INFO: `/v3/projects/692d82e02ad043f5a38de17f754a8136` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:24.431340 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4564e677-5d99-4cdd-bd79-bbb23f126109 admin admin] RBAC: Authorizing `identity:delete_project(project_id=692d82e02ad043f5a38de17f754a8136)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:24.433355 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4564e677-5d99-4cdd-bd79-bbb23f126109 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:24.517190 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:24.539542 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-4564e677-5d99-4cdd-bd79-bbb23f126109 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '692d82e02ad043f5a38de17f754a8136', 'request_id': 'req-4564e677-5d99-4cdd-bd79-bbb23f126109'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:24.546643 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-4564e677-5d99-4cdd-bd79-bbb23f126109 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '692d82e02ad043f5a38de17f754a8136', 'request_id': 'req-4564e677-5d99-4cdd-bd79-bbb23f126109'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:24.557169 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-4564e677-5d99-4cdd-bd79-bbb23f126109 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '692d82e02ad043f5a38de17f754a8136', 'request_id': 'req-4564e677-5d99-4cdd-bd79-bbb23f126109'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:24.640624 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-4564e677-5d99-4cdd-bd79-bbb23f126109 admin admin] The token cache is being invalidate because project 692d82e02ad043f5a38de17f754a8136 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:28:24.640922 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-4564e677-5d99-4cdd-bd79-bbb23f126109 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:24.641597 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-4564e677-5d99-4cdd-bd79-bbb23f126109 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:24.643168 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1017/2005] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:28:24 2022] DELETE /identity/v3/projects/692d82e02ad043f5a38de17f754a8136 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:24.647433 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70835009-4ef9-4b2a-992a-84f45c4f4177 service neutron] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:24.659716 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70835009-4ef9-4b2a-992a-84f45c4f4177 service neutron] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:24.720631 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f79f41-9fed-4570-bfbc-c0216572bc9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:24.721527 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f79f41-9fed-4570-bfbc-c0216572bc9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:24.721812 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f79f41-9fed-4570-bfbc-c0216572bc9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:24.721812 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f79f41-9fed-4570-bfbc-c0216572bc9b admin admin] PATH_INFO: `/v3/users/15d00d219f014f3c80d1f9d3331470ec` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:24.724781 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f79f41-9fed-4570-bfbc-c0216572bc9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=15d00d219f014f3c80d1f9d3331470ec)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:24.727091 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f79f41-9fed-4570-bfbc-c0216572bc9b admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:24.949511 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-25f79f41-9fed-4570-bfbc-c0216572bc9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15d00d219f014f3c80d1f9d3331470ec'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:24.963211 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-25f79f41-9fed-4570-bfbc-c0216572bc9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15d00d219f014f3c80d1f9d3331470ec'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:24.964070 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-25f79f41-9fed-4570-bfbc-c0216572bc9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15d00d219f014f3c80d1f9d3331470ec'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:24.972086 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-25f79f41-9fed-4570-bfbc-c0216572bc9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15d00d219f014f3c80d1f9d3331470ec'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:24.974019 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-25f79f41-9fed-4570-bfbc-c0216572bc9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15d00d219f014f3c80d1f9d3331470ec'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:25.059326 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 989/2006] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:28:24 2022] DELETE /identity/v3/users/15d00d219f014f3c80d1f9d3331470ec => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:25.577685 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4564e677-5d99-4cdd-bd79-bbb23f126109 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:25.579341 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-4564e677-5d99-4cdd-bd79-bbb23f126109 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:25.619956 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4564e677-5d99-4cdd-bd79-bbb23f126109 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:25.700890 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b81bbee-4c9c-44ca-8166-def60c9db479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:25.701836 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b81bbee-4c9c-44ca-8166-def60c9db479 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:25.702136 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b81bbee-4c9c-44ca-8166-def60c9db479 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:25.702414 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b81bbee-4c9c-44ca-8166-def60c9db479 admin admin] PATH_INFO: `/v3/projects/9da2a8ec241e49b6bfe1143358d22b85` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:25.703741 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b81bbee-4c9c-44ca-8166-def60c9db479 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9da2a8ec241e49b6bfe1143358d22b85)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:25.706018 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b81bbee-4c9c-44ca-8166-def60c9db479 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:25.842496 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-9b81bbee-4c9c-44ca-8166-def60c9db479 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9da2a8ec241e49b6bfe1143358d22b85', 'request_id': 'req-9b81bbee-4c9c-44ca-8166-def60c9db479'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:25.848685 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-9b81bbee-4c9c-44ca-8166-def60c9db479 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9da2a8ec241e49b6bfe1143358d22b85', 'request_id': 'req-9b81bbee-4c9c-44ca-8166-def60c9db479'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:25.854335 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-9b81bbee-4c9c-44ca-8166-def60c9db479 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9da2a8ec241e49b6bfe1143358d22b85', 'request_id': 'req-9b81bbee-4c9c-44ca-8166-def60c9db479'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:25.932308 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-9b81bbee-4c9c-44ca-8166-def60c9db479 admin admin] The token cache is being invalidate because project 9da2a8ec241e49b6bfe1143358d22b85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:28:25.932590 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-9b81bbee-4c9c-44ca-8166-def60c9db479 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:25.933297 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-9b81bbee-4c9c-44ca-8166-def60c9db479 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:25.934869 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1018/2007] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:28:25 2022] DELETE /identity/v3/projects/9da2a8ec241e49b6bfe1143358d22b85 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:25.940927 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe60b4-118e-470d-9b4f-13673fe19854 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:25.941450 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe60b4-118e-470d-9b4f-13673fe19854 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:25.941580 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe60b4-118e-470d-9b4f-13673fe19854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:25.960576 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-5dbe60b4-118e-470d-9b4f-13673fe19854 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:25.961808 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-5dbe60b4-118e-470d-9b4f-13673fe19854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:26.000464 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:26.028066 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 990/2008] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:28:26.036082 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faa4017-0bef-415d-9cbf-c086397496d0 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:26.036372 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faa4017-0bef-415d-9cbf-c086397496d0 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:26.036707 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faa4017-0bef-415d-9cbf-c086397496d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:26.057020 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-0faa4017-0bef-415d-9cbf-c086397496d0 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:26.058311 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-0faa4017-0bef-415d-9cbf-c086397496d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:26.066608 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1019/2009] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:28:26.071014 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbe60b4-118e-470d-9b4f-13673fe19854 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:26.080895 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbe60b4-118e-470d-9b4f-13673fe19854 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:26.085844 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4911d83a-c011-43d0-bb99-9d1e3bd18b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:26.086529 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4911d83a-c011-43d0-bb99-9d1e3bd18b3b admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:26.086999 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4911d83a-c011-43d0-bb99-9d1e3bd18b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:26.087686 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4911d83a-c011-43d0-bb99-9d1e3bd18b3b admin admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:26.088577 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4911d83a-c011-43d0-bb99-9d1e3bd18b3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:28:26.088893 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4911d83a-c011-43d0-bb99-9d1e3bd18b3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:26.092192 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4911d83a-c011-43d0-bb99-9d1e3bd18b3b admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:26.104437 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 991/2010] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:28:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:28:26.109701 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faa4017-0bef-415d-9cbf-c086397496d0 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:26.119449 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faa4017-0bef-415d-9cbf-c086397496d0 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:26.124710 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58b1b1a-315e-4871-b068-a368a1cbd56b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:26.125382 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b1b1a-315e-4871-b068-a368a1cbd56b admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:26.125805 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b1b1a-315e-4871-b068-a368a1cbd56b admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:26.125921 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b1b1a-315e-4871-b068-a368a1cbd56b admin admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:26.126624 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58b1b1a-315e-4871-b068-a368a1cbd56b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:26.128536 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58b1b1a-315e-4871-b068-a368a1cbd56b admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:26.220515 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1020/2011] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:28:26 2022] POST /identity/v3/projects => generated 383 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:28:26.225416 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4911d83a-c011-43d0-bb99-9d1e3bd18b3b admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:26.251276 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4911d83a-c011-43d0-bb99-9d1e3bd18b3b admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:26.319380 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0cf1a8-81ef-44a8-9b0c-d1daab35cd95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:26.320074 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0cf1a8-81ef-44a8-9b0c-d1daab35cd95 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:26.320360 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0cf1a8-81ef-44a8-9b0c-d1daab35cd95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:26.320962 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0cf1a8-81ef-44a8-9b0c-d1daab35cd95 admin admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:26.321694 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0cf1a8-81ef-44a8-9b0c-d1daab35cd95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:26.324340 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0cf1a8-81ef-44a8-9b0c-d1daab35cd95 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:26.400997 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 992/2012] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:28:26 2022] POST /identity/v3/users => generated 292 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:28:26.406131 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58b1b1a-315e-4871-b068-a368a1cbd56b admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:26.416966 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58b1b1a-315e-4871-b068-a368a1cbd56b admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:26.422374 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5766f6-2036-41aa-8dbe-9bcc60379b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:26.423058 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5766f6-2036-41aa-8dbe-9bcc60379b63 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:26.423344 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5766f6-2036-41aa-8dbe-9bcc60379b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:26.423622 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5766f6-2036-41aa-8dbe-9bcc60379b63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:26.424356 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5766f6-2036-41aa-8dbe-9bcc60379b63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:26.426154 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5766f6-2036-41aa-8dbe-9bcc60379b63 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:26.438400 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1021/2013] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:28:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:28:26.441863 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0cf1a8-81ef-44a8-9b0c-d1daab35cd95 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:26.453972 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0cf1a8-81ef-44a8-9b0c-d1daab35cd95 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:26.459364 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c79eea-3166-499e-b9eb-5f7eaa424e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:26.460053 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c79eea-3166-499e-b9eb-5f7eaa424e9e admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:26.460344 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c79eea-3166-499e-b9eb-5f7eaa424e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:26.460651 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c79eea-3166-499e-b9eb-5f7eaa424e9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:26.461459 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c79eea-3166-499e-b9eb-5f7eaa424e9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:26.463065 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c79eea-3166-499e-b9eb-5f7eaa424e9e admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:26.478646 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 993/2014] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:28:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:28:26.480071 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5766f6-2036-41aa-8dbe-9bcc60379b63 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:26.492717 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5766f6-2036-41aa-8dbe-9bcc60379b63 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:26.498560 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f6dc68-9c8d-41cc-b354-e750f3a91048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:26.499733 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f6dc68-9c8d-41cc-b354-e750f3a91048 admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:26.500018 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f6dc68-9c8d-41cc-b354-e750f3a91048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:26.500295 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f6dc68-9c8d-41cc-b354-e750f3a91048 admin admin] PATH_INFO: `/v3/projects/c8c579dbb9974e6fafd6d45e3c8ff3b2/users/57042cafffe14d1da8559a7a304ce124/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:26.504973 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:26.518613 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f6dc68-9c8d-41cc-b354-e750f3a91048 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8c579dbb9974e6fafd6d45e3c8ff3b2, user_id=57042cafffe14d1da8559a7a304ce124, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:26.522026 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f6dc68-9c8d-41cc-b354-e750f3a91048 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:26.626088 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1022/2015] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:28:26 2022] PUT /identity/v3/projects/c8c579dbb9974e6fafd6d45e3c8ff3b2/users/57042cafffe14d1da8559a7a304ce124/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:26.637962 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3956a3cb-88d1-4802-9489-48e43cff627e None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:26.638358 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3956a3cb-88d1-4802-9489-48e43cff627e None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:26.638714 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3956a3cb-88d1-4802-9489-48e43cff627e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:26.682368 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-3956a3cb-88d1-4802-9489-48e43cff627e None None] MFA Rules not processed for user `57042cafffe14d1da8559a7a304ce124`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:26.686010 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-3956a3cb-88d1-4802-9489-48e43cff627e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:26.792685 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 994/2016] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:28:26.794098 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f6dc68-9c8d-41cc-b354-e750f3a91048 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:26.882746 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f6dc68-9c8d-41cc-b354-e750f3a91048 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:26.887905 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd1bdb3-bf73-4fa4-8b48-3c2092c14d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:26.888653 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd1bdb3-bf73-4fa4-8b48-3c2092c14d2b admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:26.888928 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd1bdb3-bf73-4fa4-8b48-3c2092c14d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:26.889208 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd1bdb3-bf73-4fa4-8b48-3c2092c14d2b admin admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:26.890420 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd1bdb3-bf73-4fa4-8b48-3c2092c14d2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:26.891839 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd1bdb3-bf73-4fa4-8b48-3c2092c14d2b admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:26.968770 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1023/2017] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:28:26 2022] POST /identity/v3/projects => generated 381 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:28:26.973084 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3956a3cb-88d1-4802-9489-48e43cff627e None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:26.986509 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3956a3cb-88d1-4802-9489-48e43cff627e None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:27.019760 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:27.046475 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d92556-41bf-4d81-9c49-8530d9b61ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:27.047492 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d92556-41bf-4d81-9c49-8530d9b61ae4 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:27.048121 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d92556-41bf-4d81-9c49-8530d9b61ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:27.048426 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d92556-41bf-4d81-9c49-8530d9b61ae4 admin admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:27.049202 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d92556-41bf-4d81-9c49-8530d9b61ae4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:27.051129 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d92556-41bf-4d81-9c49-8530d9b61ae4 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:27.166027 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 995/2018] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:28:26 2022] POST /identity/v3/users => generated 297 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:28:27.170836 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd1bdb3-bf73-4fa4-8b48-3c2092c14d2b admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:27.181398 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd1bdb3-bf73-4fa4-8b48-3c2092c14d2b admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:27.186944 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a329ff-15b1-47d1-8068-300dc087dd74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:27.187658 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a329ff-15b1-47d1-8068-300dc087dd74 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:27.187947 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a329ff-15b1-47d1-8068-300dc087dd74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:27.188228 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a329ff-15b1-47d1-8068-300dc087dd74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:27.189004 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a329ff-15b1-47d1-8068-300dc087dd74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:27.190636 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a329ff-15b1-47d1-8068-300dc087dd74 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:27.202537 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1024/2019] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:28:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:28:27.206931 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d92556-41bf-4d81-9c49-8530d9b61ae4 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:27.217526 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d92556-41bf-4d81-9c49-8530d9b61ae4 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:27.222147 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edcc426-1392-4f47-b9bf-374199d7fa0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:27.222962 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edcc426-1392-4f47-b9bf-374199d7fa0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:27.223240 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edcc426-1392-4f47-b9bf-374199d7fa0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:27.223513 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edcc426-1392-4f47-b9bf-374199d7fa0a admin admin] PATH_INFO: `/v3/projects/45c23b318af34be2b7dd454ff3c8665d/users/4fd44e739f7b4584bda3ea17a50af702/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:27.238969 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edcc426-1392-4f47-b9bf-374199d7fa0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=45c23b318af34be2b7dd454ff3c8665d, user_id=4fd44e739f7b4584bda3ea17a50af702, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:27.241733 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edcc426-1392-4f47-b9bf-374199d7fa0a admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:27.310273 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 996/2020] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:28:27 2022] PUT /identity/v3/projects/45c23b318af34be2b7dd454ff3c8665d/users/4fd44e739f7b4584bda3ea17a50af702/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:27.314412 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a329ff-15b1-47d1-8068-300dc087dd74 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:27.324741 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a329ff-15b1-47d1-8068-300dc087dd74 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:27.382612 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6e9f0a-8bb2-405b-88b9-a05d957d6f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:27.383288 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6e9f0a-8bb2-405b-88b9-a05d957d6f86 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:27.383577 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6e9f0a-8bb2-405b-88b9-a05d957d6f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:27.383855 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6e9f0a-8bb2-405b-88b9-a05d957d6f86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:27.384647 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6e9f0a-8bb2-405b-88b9-a05d957d6f86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:27.386295 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6e9f0a-8bb2-405b-88b9-a05d957d6f86 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:27.396234 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1025/2021] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:28:27 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:28:27.400462 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edcc426-1392-4f47-b9bf-374199d7fa0a admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:27.411340 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edcc426-1392-4f47-b9bf-374199d7fa0a admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:27.416376 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44ed404-f938-44aa-9eae-b33d5e923355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:27.417970 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ed404-f938-44aa-9eae-b33d5e923355 admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:27.418279 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ed404-f938-44aa-9eae-b33d5e923355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:27.418565 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ed404-f938-44aa-9eae-b33d5e923355 admin admin] PATH_INFO: `/v3/projects/45c23b318af34be2b7dd454ff3c8665d/users/4fd44e739f7b4584bda3ea17a50af702/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:27.420667 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44ed404-f938-44aa-9eae-b33d5e923355 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45c23b318af34be2b7dd454ff3c8665d, user_id=4fd44e739f7b4584bda3ea17a50af702, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:27.423353 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44ed404-f938-44aa-9eae-b33d5e923355 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:27.434432 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 997/2022] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:28:27 2022] PUT /identity/v3/projects/45c23b318af34be2b7dd454ff3c8665d/users/4fd44e739f7b4584bda3ea17a50af702/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:27.444717 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82dca3d-7739-4914-a3bb-7cb16486ff92 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:27.445011 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82dca3d-7739-4914-a3bb-7cb16486ff92 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:27.445292 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82dca3d-7739-4914-a3bb-7cb16486ff92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:27.473042 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-c82dca3d-7739-4914-a3bb-7cb16486ff92 None None] MFA Rules not processed for user `4fd44e739f7b4584bda3ea17a50af702`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:27.474207 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-c82dca3d-7739-4914-a3bb-7cb16486ff92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:27.519954 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:27.546808 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1026/2023] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:27 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:28:27.557505 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44ed404-f938-44aa-9eae-b33d5e923355 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:27.559623 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-e44ed404-f938-44aa-9eae-b33d5e923355 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:27.619603 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44ed404-f938-44aa-9eae-b33d5e923355 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:27.623306 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431c5042-3b6b-4ac8-8d9d-5725841adab0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:27.623957 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431c5042-3b6b-4ac8-8d9d-5725841adab0 service cinder] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:27.624613 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431c5042-3b6b-4ac8-8d9d-5725841adab0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:27.625013 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431c5042-3b6b-4ac8-8d9d-5725841adab0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:27.634469 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431c5042-3b6b-4ac8-8d9d-5725841adab0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:27.635961 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431c5042-3b6b-4ac8-8d9d-5725841adab0 service cinder] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:27.645934 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 998/2024] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:28:27 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:28:27.745693 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82dca3d-7739-4914-a3bb-7cb16486ff92 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:27.747103 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-c82dca3d-7739-4914-a3bb-7cb16486ff92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:27.764137 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82dca3d-7739-4914-a3bb-7cb16486ff92 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:27.766613 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0e98fd-8ebd-4e8a-a450-f7d3c7d41308 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:27.767343 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e98fd-8ebd-4e8a-a450-f7d3c7d41308 service cinder] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:27.767745 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e98fd-8ebd-4e8a-a450-f7d3c7d41308 service cinder] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:27.768095 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e98fd-8ebd-4e8a-a450-f7d3c7d41308 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:27.838489 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0e98fd-8ebd-4e8a-a450-f7d3c7d41308 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:27.840822 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0e98fd-8ebd-4e8a-a450-f7d3c7d41308 service cinder] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:27.852638 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1027/2025] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:28:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:28:33.110019 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0da81-bf8a-4327-bca6-215e19eceebc None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:33.110545 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0da81-bf8a-4327-bca6-215e19eceebc None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:33.110640 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0da81-bf8a-4327-bca6-215e19eceebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:33.129487 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-1cb0da81-bf8a-4327-bca6-215e19eceebc None None] MFA Rules not processed for user `b6bbd45ee6964c4986f3c778ab8ad231`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:33.130840 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-1cb0da81-bf8a-4327-bca6-215e19eceebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:33.195539 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 999/2026] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:28:34.082696 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0e98fd-8ebd-4e8a-a450-f7d3c7d41308 service cinder] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:34.149009 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0e98fd-8ebd-4e8a-a450-f7d3c7d41308 service cinder] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:34.155280 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa3eb76-4d21-4edc-ab07-820c1926dbe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:34.156243 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3eb76-4d21-4edc-ab07-820c1926dbe6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:34.156634 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3eb76-4d21-4edc-ab07-820c1926dbe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:34.157185 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3eb76-4d21-4edc-ab07-820c1926dbe6 admin admin] PATH_INFO: `/v3/users/57042cafffe14d1da8559a7a304ce124` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:34.158415 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa3eb76-4d21-4edc-ab07-820c1926dbe6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57042cafffe14d1da8559a7a304ce124)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:34.160494 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa3eb76-4d21-4edc-ab07-820c1926dbe6 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:34.376008 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-5aa3eb76-4d21-4edc-ab07-820c1926dbe6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57042cafffe14d1da8559a7a304ce124'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:34.390867 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-5aa3eb76-4d21-4edc-ab07-820c1926dbe6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57042cafffe14d1da8559a7a304ce124'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:34.451989 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-5aa3eb76-4d21-4edc-ab07-820c1926dbe6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57042cafffe14d1da8559a7a304ce124'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:34.452294 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-5aa3eb76-4d21-4edc-ab07-820c1926dbe6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57042cafffe14d1da8559a7a304ce124'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:34.462659 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-5aa3eb76-4d21-4edc-ab07-820c1926dbe6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57042cafffe14d1da8559a7a304ce124'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:34.464923 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:34.470170 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1028/2027] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:28:34 2022] DELETE /identity/v3/users/57042cafffe14d1da8559a7a304ce124 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:34.474956 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a521d4b6-259f-45fe-a0a0-452176b36ccd None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:34.475449 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a521d4b6-259f-45fe-a0a0-452176b36ccd None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:34.475581 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a521d4b6-259f-45fe-a0a0-452176b36ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:34.498533 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-a521d4b6-259f-45fe-a0a0-452176b36ccd None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:34.506479 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-a521d4b6-259f-45fe-a0a0-452176b36ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:34.536258 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:34.568009 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1000/2028] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:28:34.578381 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa3eb76-4d21-4edc-ab07-820c1926dbe6 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:34.579793 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-5aa3eb76-4d21-4edc-ab07-820c1926dbe6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:34.640484 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa3eb76-4d21-4edc-ab07-820c1926dbe6 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:34.645348 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d139e-7846-4854-9b0f-792e3de88db1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:34.646036 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d139e-7846-4854-9b0f-792e3de88db1 service neutron] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:34.646315 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d139e-7846-4854-9b0f-792e3de88db1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:34.646609 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d139e-7846-4854-9b0f-792e3de88db1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:34.657063 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68d139e-7846-4854-9b0f-792e3de88db1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:34.658627 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68d139e-7846-4854-9b0f-792e3de88db1 service neutron] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:34.668760 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1029/2029] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:28:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:28:35.192117 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a521d4b6-259f-45fe-a0a0-452176b36ccd None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:35.193900 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-a521d4b6-259f-45fe-a0a0-452176b36ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:35.223092 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a521d4b6-259f-45fe-a0a0-452176b36ccd None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:35.228786 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79e19c2-e720-4cf4-93da-eb70b18b0117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:35.230423 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e19c2-e720-4cf4-93da-eb70b18b0117 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:35.230708 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e19c2-e720-4cf4-93da-eb70b18b0117 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:35.231055 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e19c2-e720-4cf4-93da-eb70b18b0117 admin admin] PATH_INFO: `/v3/projects/c8c579dbb9974e6fafd6d45e3c8ff3b2` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:35.232278 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79e19c2-e720-4cf4-93da-eb70b18b0117 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8c579dbb9974e6fafd6d45e3c8ff3b2)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:35.234327 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79e19c2-e720-4cf4-93da-eb70b18b0117 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:35.319344 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-d79e19c2-e720-4cf4-93da-eb70b18b0117 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8c579dbb9974e6fafd6d45e3c8ff3b2', 'request_id': 'req-d79e19c2-e720-4cf4-93da-eb70b18b0117'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:35.371876 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-d79e19c2-e720-4cf4-93da-eb70b18b0117 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8c579dbb9974e6fafd6d45e3c8ff3b2', 'request_id': 'req-d79e19c2-e720-4cf4-93da-eb70b18b0117'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:35.378495 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-d79e19c2-e720-4cf4-93da-eb70b18b0117 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8c579dbb9974e6fafd6d45e3c8ff3b2', 'request_id': 'req-d79e19c2-e720-4cf4-93da-eb70b18b0117'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:35.390584 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-d79e19c2-e720-4cf4-93da-eb70b18b0117 admin admin] The token cache is being invalidate because project c8c579dbb9974e6fafd6d45e3c8ff3b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:28:35.390869 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-d79e19c2-e720-4cf4-93da-eb70b18b0117 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:35.391525 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-d79e19c2-e720-4cf4-93da-eb70b18b0117 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:35.393169 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1001/2030] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:28:35 2022] DELETE /identity/v3/projects/c8c579dbb9974e6fafd6d45e3c8ff3b2 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:35.397303 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d139e-7846-4854-9b0f-792e3de88db1 service neutron] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:35.408209 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d139e-7846-4854-9b0f-792e3de88db1 service neutron] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:35.465527 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94591ad-baeb-4ffb-97cf-38c091ccb7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:35.466409 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94591ad-baeb-4ffb-97cf-38c091ccb7fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:35.466628 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94591ad-baeb-4ffb-97cf-38c091ccb7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:35.466829 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94591ad-baeb-4ffb-97cf-38c091ccb7fb admin admin] PATH_INFO: `/v3/users/4fd44e739f7b4584bda3ea17a50af702` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:35.468012 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94591ad-baeb-4ffb-97cf-38c091ccb7fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fd44e739f7b4584bda3ea17a50af702)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:35.472511 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94591ad-baeb-4ffb-97cf-38c091ccb7fb admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:35.694801 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-d94591ad-baeb-4ffb-97cf-38c091ccb7fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fd44e739f7b4584bda3ea17a50af702'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:35.708304 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-d94591ad-baeb-4ffb-97cf-38c091ccb7fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fd44e739f7b4584bda3ea17a50af702'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:35.806199 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-d94591ad-baeb-4ffb-97cf-38c091ccb7fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fd44e739f7b4584bda3ea17a50af702'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:35.806796 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-d94591ad-baeb-4ffb-97cf-38c091ccb7fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fd44e739f7b4584bda3ea17a50af702'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:35.815174 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-d94591ad-baeb-4ffb-97cf-38c091ccb7fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fd44e739f7b4584bda3ea17a50af702'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:35.821556 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1030/2031] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:28:35 2022] DELETE /identity/v3/users/4fd44e739f7b4584bda3ea17a50af702 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:36.347724 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79e19c2-e720-4cf4-93da-eb70b18b0117 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:36.347724 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-d79e19c2-e720-4cf4-93da-eb70b18b0117 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:36.360768 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79e19c2-e720-4cf4-93da-eb70b18b0117 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:36.420987 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd96467-f72a-4fb7-b2f4-2d359e59e54f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:36.421851 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd96467-f72a-4fb7-b2f4-2d359e59e54f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:36.422132 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd96467-f72a-4fb7-b2f4-2d359e59e54f admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:36.422459 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd96467-f72a-4fb7-b2f4-2d359e59e54f admin admin] PATH_INFO: `/v3/projects/45c23b318af34be2b7dd454ff3c8665d` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:36.423592 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd96467-f72a-4fb7-b2f4-2d359e59e54f admin admin] RBAC: Authorizing `identity:delete_project(project_id=45c23b318af34be2b7dd454ff3c8665d)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:36.425532 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd96467-f72a-4fb7-b2f4-2d359e59e54f admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:36.514850 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-6dd96467-f72a-4fb7-b2f4-2d359e59e54f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45c23b318af34be2b7dd454ff3c8665d', 'request_id': 'req-6dd96467-f72a-4fb7-b2f4-2d359e59e54f'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:36.583057 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-6dd96467-f72a-4fb7-b2f4-2d359e59e54f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45c23b318af34be2b7dd454ff3c8665d', 'request_id': 'req-6dd96467-f72a-4fb7-b2f4-2d359e59e54f'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:36.587860 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:36.591026 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-6dd96467-f72a-4fb7-b2f4-2d359e59e54f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45c23b318af34be2b7dd454ff3c8665d', 'request_id': 'req-6dd96467-f72a-4fb7-b2f4-2d359e59e54f'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:36.602595 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-6dd96467-f72a-4fb7-b2f4-2d359e59e54f admin admin] The token cache is being invalidate because project 45c23b318af34be2b7dd454ff3c8665d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:28:36.602887 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-6dd96467-f72a-4fb7-b2f4-2d359e59e54f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:36.603587 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-6dd96467-f72a-4fb7-b2f4-2d359e59e54f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:36.606135 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1002/2032] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:28:36 2022] DELETE /identity/v3/projects/45c23b318af34be2b7dd454ff3c8665d => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:36.611696 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77834756-a559-4516-a580-852e9833e2ad None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:36.615048 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77834756-a559-4516-a580-852e9833e2ad None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:36.615048 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77834756-a559-4516-a580-852e9833e2ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:36.631515 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-77834756-a559-4516-a580-852e9833e2ad None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:36.632782 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-77834756-a559-4516-a580-852e9833e2ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:36.646780 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:36.696432 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1031/2033] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:28:36.705307 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7343467e-368f-4e1f-8a46-3faf76f0e34a None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:36.705601 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7343467e-368f-4e1f-8a46-3faf76f0e34a None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:36.705884 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7343467e-368f-4e1f-8a46-3faf76f0e34a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:36.725777 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-7343467e-368f-4e1f-8a46-3faf76f0e34a None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:36.727002 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-7343467e-368f-4e1f-8a46-3faf76f0e34a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:36.736907 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1003/2034] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:28:36.741490 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77834756-a559-4516-a580-852e9833e2ad None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:36.752276 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77834756-a559-4516-a580-852e9833e2ad None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:36.757196 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2796ebc-ac62-4e8a-8c98-a361a8f62436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:36.757963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2796ebc-ac62-4e8a-8c98-a361a8f62436 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:36.758258 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2796ebc-ac62-4e8a-8c98-a361a8f62436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:36.758541 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2796ebc-ac62-4e8a-8c98-a361a8f62436 admin admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:36.759373 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2796ebc-ac62-4e8a-8c98-a361a8f62436 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:28:36.759903 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2796ebc-ac62-4e8a-8c98-a361a8f62436 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:36.762303 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2796ebc-ac62-4e8a-8c98-a361a8f62436 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:36.774347 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1032/2035] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:28:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:28:36.789369 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7343467e-368f-4e1f-8a46-3faf76f0e34a None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:36.799627 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7343467e-368f-4e1f-8a46-3faf76f0e34a None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:36.804357 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c1c07b-42cc-48a8-8fd7-004fb94798a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:36.805055 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c1c07b-42cc-48a8-8fd7-004fb94798a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:36.805346 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c1c07b-42cc-48a8-8fd7-004fb94798a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:36.805625 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c1c07b-42cc-48a8-8fd7-004fb94798a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:36.806332 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c1c07b-42cc-48a8-8fd7-004fb94798a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:36.808563 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c1c07b-42cc-48a8-8fd7-004fb94798a9 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:36.890385 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1004/2036] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:28:36 2022] POST /identity/v3/projects => generated 367 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:28:36.894085 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2796ebc-ac62-4e8a-8c98-a361a8f62436 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:36.904900 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2796ebc-ac62-4e8a-8c98-a361a8f62436 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:36.966382 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b66103-2724-4283-bf56-5c2ac87e43ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:36.967044 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b66103-2724-4283-bf56-5c2ac87e43ef admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:36.967347 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b66103-2724-4283-bf56-5c2ac87e43ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:36.967619 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b66103-2724-4283-bf56-5c2ac87e43ef admin admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:36.968317 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b66103-2724-4283-bf56-5c2ac87e43ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:36.970232 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b66103-2724-4283-bf56-5c2ac87e43ef admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:37.057441 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1033/2037] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:28:36 2022] POST /identity/v3/users => generated 284 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:28:37.061588 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c1c07b-42cc-48a8-8fd7-004fb94798a9 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:37.071817 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c1c07b-42cc-48a8-8fd7-004fb94798a9 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:37.077162 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2a4541-b23f-4a29-ab95-6ea77ed93424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:37.077819 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a4541-b23f-4a29-ab95-6ea77ed93424 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:37.078101 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a4541-b23f-4a29-ab95-6ea77ed93424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:37.078378 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a4541-b23f-4a29-ab95-6ea77ed93424 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:37.079113 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2a4541-b23f-4a29-ab95-6ea77ed93424 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:37.080726 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2a4541-b23f-4a29-ab95-6ea77ed93424 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:37.091758 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1005/2038] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:28:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:28:37.095838 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b66103-2724-4283-bf56-5c2ac87e43ef admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:37.106050 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b66103-2724-4283-bf56-5c2ac87e43ef admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:37.110673 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e7e88b-7733-4468-afa3-7e6722aa3f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:37.111334 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e7e88b-7733-4468-afa3-7e6722aa3f47 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:37.111617 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e7e88b-7733-4468-afa3-7e6722aa3f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:37.111887 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e7e88b-7733-4468-afa3-7e6722aa3f47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:37.112663 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e7e88b-7733-4468-afa3-7e6722aa3f47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:37.114238 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e7e88b-7733-4468-afa3-7e6722aa3f47 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:37.124349 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1034/2039] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:28:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:28:37.128508 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2a4541-b23f-4a29-ab95-6ea77ed93424 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:37.139644 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2a4541-b23f-4a29-ab95-6ea77ed93424 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:37.144150 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47940112-8b54-4ee1-9522-f26386ed1f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:37.145066 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47940112-8b54-4ee1-9522-f26386ed1f0d admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:37.145344 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47940112-8b54-4ee1-9522-f26386ed1f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:37.145618 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47940112-8b54-4ee1-9522-f26386ed1f0d admin admin] PATH_INFO: `/v3/projects/20eedaa8dc5e474fa7b74bd4b75b95b2/users/2d101bbf45654628b9f2d780bc13fcbf/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:37.161098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47940112-8b54-4ee1-9522-f26386ed1f0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=20eedaa8dc5e474fa7b74bd4b75b95b2, user_id=2d101bbf45654628b9f2d780bc13fcbf, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:37.163715 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47940112-8b54-4ee1-9522-f26386ed1f0d admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:37.247533 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1006/2040] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:28:37 2022] PUT /identity/v3/projects/20eedaa8dc5e474fa7b74bd4b75b95b2/users/2d101bbf45654628b9f2d780bc13fcbf/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:37.260143 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf25404-e5e6-4785-bdb1-b11bf4d969b3 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:37.260597 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf25404-e5e6-4785-bdb1-b11bf4d969b3 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:37.261051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf25404-e5e6-4785-bdb1-b11bf4d969b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:37.293681 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-bdf25404-e5e6-4785-bdb1-b11bf4d969b3 None None] MFA Rules not processed for user `2d101bbf45654628b9f2d780bc13fcbf`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:37.294441 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-bdf25404-e5e6-4785-bdb1-b11bf4d969b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:37.379267 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1035/2041] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:37 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:28:37.384700 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47940112-8b54-4ee1-9522-f26386ed1f0d admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:37.452827 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47940112-8b54-4ee1-9522-f26386ed1f0d admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:37.457664 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d6bb52-ad8a-4170-964e-1d7f086554b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:37.458340 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d6bb52-ad8a-4170-964e-1d7f086554b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:37.458633 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d6bb52-ad8a-4170-964e-1d7f086554b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:37.458896 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d6bb52-ad8a-4170-964e-1d7f086554b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:37.459609 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d6bb52-ad8a-4170-964e-1d7f086554b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:37.461551 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d6bb52-ad8a-4170-964e-1d7f086554b9 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:37.538693 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1007/2042] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:28:37 2022] POST /identity/v3/projects => generated 367 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:28:37.543689 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf25404-e5e6-4785-bdb1-b11bf4d969b3 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:37.554869 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf25404-e5e6-4785-bdb1-b11bf4d969b3 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:37.566418 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:37.613616 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405c208b-e605-4f37-9930-1a81de7e2971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:37.614281 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c208b-e605-4f37-9930-1a81de7e2971 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:37.614565 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c208b-e605-4f37-9930-1a81de7e2971 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:37.614843 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c208b-e605-4f37-9930-1a81de7e2971 admin admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:37.615561 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405c208b-e605-4f37-9930-1a81de7e2971 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:37.617620 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405c208b-e605-4f37-9930-1a81de7e2971 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:37.684519 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1036/2043] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:28:37 2022] POST /identity/v3/users => generated 290 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:28:37.686649 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d6bb52-ad8a-4170-964e-1d7f086554b9 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:37.698188 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d6bb52-ad8a-4170-964e-1d7f086554b9 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:37.703312 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c393382-d1fd-484c-8c9a-c32391647873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:37.704027 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c393382-d1fd-484c-8c9a-c32391647873 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:37.704315 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c393382-d1fd-484c-8c9a-c32391647873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:37.704632 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c393382-d1fd-484c-8c9a-c32391647873 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:37.705399 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c393382-d1fd-484c-8c9a-c32391647873 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:37.707135 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c393382-d1fd-484c-8c9a-c32391647873 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:37.719082 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1008/2044] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:28:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:28:37.723212 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405c208b-e605-4f37-9930-1a81de7e2971 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:37.734526 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405c208b-e605-4f37-9930-1a81de7e2971 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:37.739278 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6672ac-f53b-4983-b8e7-04ab3b194768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:37.740173 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6672ac-f53b-4983-b8e7-04ab3b194768 admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:37.740486 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6672ac-f53b-4983-b8e7-04ab3b194768 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:37.740767 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6672ac-f53b-4983-b8e7-04ab3b194768 admin admin] PATH_INFO: `/v3/projects/583b69cd118e4998b2f23331eff9b9ac/users/f32f185c3afd45fc8295987622b6d17f/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:37.757328 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6672ac-f53b-4983-b8e7-04ab3b194768 admin admin] RBAC: Authorizing `identity:create_grant(project_id=583b69cd118e4998b2f23331eff9b9ac, user_id=f32f185c3afd45fc8295987622b6d17f, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:37.762806 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6672ac-f53b-4983-b8e7-04ab3b194768 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:37.840339 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1037/2045] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:28:37 2022] PUT /identity/v3/projects/583b69cd118e4998b2f23331eff9b9ac/users/f32f185c3afd45fc8295987622b6d17f/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:37.844588 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c393382-d1fd-484c-8c9a-c32391647873 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:37.855803 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c393382-d1fd-484c-8c9a-c32391647873 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:37.918530 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bfa6d1-23dc-4e98-a371-c30e3c4655f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:37.919276 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bfa6d1-23dc-4e98-a371-c30e3c4655f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:37.919566 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bfa6d1-23dc-4e98-a371-c30e3c4655f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:37.919856 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bfa6d1-23dc-4e98-a371-c30e3c4655f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:37.920633 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bfa6d1-23dc-4e98-a371-c30e3c4655f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:37.922344 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bfa6d1-23dc-4e98-a371-c30e3c4655f9 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:37.923694 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:37.934068 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1009/2046] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:28:37 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:28:37.938408 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6672ac-f53b-4983-b8e7-04ab3b194768 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:37.951726 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6672ac-f53b-4983-b8e7-04ab3b194768 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:37.957026 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d32950-ed0d-46e2-b678-0070c015853b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:37.958698 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d32950-ed0d-46e2-b678-0070c015853b admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:37.958698 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d32950-ed0d-46e2-b678-0070c015853b admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:37.958698 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d32950-ed0d-46e2-b678-0070c015853b admin admin] PATH_INFO: `/v3/projects/583b69cd118e4998b2f23331eff9b9ac/users/f32f185c3afd45fc8295987622b6d17f/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:37.961129 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d32950-ed0d-46e2-b678-0070c015853b admin admin] RBAC: Authorizing `identity:create_grant(project_id=583b69cd118e4998b2f23331eff9b9ac, user_id=f32f185c3afd45fc8295987622b6d17f, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:37.964552 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d32950-ed0d-46e2-b678-0070c015853b admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:37.978230 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1038/2047] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:28:37 2022] PUT /identity/v3/projects/583b69cd118e4998b2f23331eff9b9ac/users/f32f185c3afd45fc8295987622b6d17f/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:28:37.989665 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a5fc99-c92d-4aed-83a6-b9c54c51b1b1 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:37.989956 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a5fc99-c92d-4aed-83a6-b9c54c51b1b1 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:37.995404 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a5fc99-c92d-4aed-83a6-b9c54c51b1b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:38.019692 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-79a5fc99-c92d-4aed-83a6-b9c54c51b1b1 None None] MFA Rules not processed for user `f32f185c3afd45fc8295987622b6d17f`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:38.021001 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-79a5fc99-c92d-4aed-83a6-b9c54c51b1b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:38.105253 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1010/2048] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:37 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:28:38.117456 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d32950-ed0d-46e2-b678-0070c015853b admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:38.119173 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-91d32950-ed0d-46e2-b678-0070c015853b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:38.181026 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d32950-ed0d-46e2-b678-0070c015853b admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:38.184994 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4336793c-405e-4b2e-993f-78bc17f936fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:38.185720 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336793c-405e-4b2e-993f-78bc17f936fc service cinder] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:38.186003 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336793c-405e-4b2e-993f-78bc17f936fc service cinder] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:38.186281 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336793c-405e-4b2e-993f-78bc17f936fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:38.197593 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4336793c-405e-4b2e-993f-78bc17f936fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:38.199157 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4336793c-405e-4b2e-993f-78bc17f936fc service cinder] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:38.210905 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1039/2049] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:28:38 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:28:38.475440 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a5fc99-c92d-4aed-83a6-b9c54c51b1b1 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:38.493489 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a5fc99-c92d-4aed-83a6-b9c54c51b1b1 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:38.506274 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99fb9c9-c789-4d6c-98c0-869219f436ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:38.506952 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99fb9c9-c789-4d6c-98c0-869219f436ef service cinder] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:38.507387 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99fb9c9-c789-4d6c-98c0-869219f436ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:38.507667 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99fb9c9-c789-4d6c-98c0-869219f436ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:38.600089 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99fb9c9-c789-4d6c-98c0-869219f436ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:38.601823 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99fb9c9-c789-4d6c-98c0-869219f436ef service cinder] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:38.612576 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1011/2050] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:28:38 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:28:44.880271 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4336793c-405e-4b2e-993f-78bc17f936fc service cinder] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:44.882037 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-4336793c-405e-4b2e-993f-78bc17f936fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:44.965918 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4336793c-405e-4b2e-993f-78bc17f936fc service cinder] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:44.971084 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ae9861-0bfb-417d-b0f1-9ef6e1ee74b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:44.972086 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae9861-0bfb-417d-b0f1-9ef6e1ee74b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:44.972400 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae9861-0bfb-417d-b0f1-9ef6e1ee74b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:44.972700 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae9861-0bfb-417d-b0f1-9ef6e1ee74b8 admin admin] PATH_INFO: `/v3/users/0548f4595850464a8e3d9c9b4f7014cf` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:44.974487 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ae9861-0bfb-417d-b0f1-9ef6e1ee74b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0548f4595850464a8e3d9c9b4f7014cf)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:44.976248 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ae9861-0bfb-417d-b0f1-9ef6e1ee74b8 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:45.003217 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:45.170021 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-87ae9861-0bfb-417d-b0f1-9ef6e1ee74b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0548f4595850464a8e3d9c9b4f7014cf'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:45.183151 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-87ae9861-0bfb-417d-b0f1-9ef6e1ee74b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0548f4595850464a8e3d9c9b4f7014cf'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:45.286088 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-87ae9861-0bfb-417d-b0f1-9ef6e1ee74b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0548f4595850464a8e3d9c9b4f7014cf'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:45.289081 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-87ae9861-0bfb-417d-b0f1-9ef6e1ee74b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0548f4595850464a8e3d9c9b4f7014cf'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:45.312475 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-87ae9861-0bfb-417d-b0f1-9ef6e1ee74b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0548f4595850464a8e3d9c9b4f7014cf'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:45.320710 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1040/2051] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:28:44 2022] DELETE /identity/v3/users/0548f4595850464a8e3d9c9b4f7014cf => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:45.325422 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cca59b-6d2c-4a78-a79d-2aa7bb9730a1 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:45.325961 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cca59b-6d2c-4a78-a79d-2aa7bb9730a1 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:45.326706 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cca59b-6d2c-4a78-a79d-2aa7bb9730a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:45.360944 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-29cca59b-6d2c-4a78-a79d-2aa7bb9730a1 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:45.362468 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-29cca59b-6d2c-4a78-a79d-2aa7bb9730a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:45.451833 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1012/2052] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:28:45.462791 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ae9861-0bfb-417d-b0f1-9ef6e1ee74b8 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:45.468486 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-87ae9861-0bfb-417d-b0f1-9ef6e1ee74b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:45.563318 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ae9861-0bfb-417d-b0f1-9ef6e1ee74b8 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:45.570998 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd57a274-4ece-4c32-8da7-70ff808ce37f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:45.572102 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd57a274-4ece-4c32-8da7-70ff808ce37f service neutron] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:45.572495 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd57a274-4ece-4c32-8da7-70ff808ce37f service neutron] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:45.573818 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd57a274-4ece-4c32-8da7-70ff808ce37f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:45.589416 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd57a274-4ece-4c32-8da7-70ff808ce37f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:45.591522 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd57a274-4ece-4c32-8da7-70ff808ce37f service neutron] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:45.606604 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1041/2053] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:28:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:28:45.718285 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cca59b-6d2c-4a78-a79d-2aa7bb9730a1 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:45.719818 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-29cca59b-6d2c-4a78-a79d-2aa7bb9730a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:45.731935 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cca59b-6d2c-4a78-a79d-2aa7bb9730a1 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:45.737254 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d8b0e9-4329-4b88-ab43-eb37a733cf77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:45.738201 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8b0e9-4329-4b88-ab43-eb37a733cf77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:45.738487 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8b0e9-4329-4b88-ab43-eb37a733cf77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:45.738782 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8b0e9-4329-4b88-ab43-eb37a733cf77 admin admin] PATH_INFO: `/v3/users/2d101bbf45654628b9f2d780bc13fcbf` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:45.740000 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d8b0e9-4329-4b88-ab43-eb37a733cf77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d101bbf45654628b9f2d780bc13fcbf)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:45.742179 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d8b0e9-4329-4b88-ab43-eb37a733cf77 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:45.758803 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:46.019090 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-11d8b0e9-4329-4b88-ab43-eb37a733cf77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d101bbf45654628b9f2d780bc13fcbf'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:46.041085 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-11d8b0e9-4329-4b88-ab43-eb37a733cf77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d101bbf45654628b9f2d780bc13fcbf'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:46.043390 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-11d8b0e9-4329-4b88-ab43-eb37a733cf77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d101bbf45654628b9f2d780bc13fcbf'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:46.053649 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-11d8b0e9-4329-4b88-ab43-eb37a733cf77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d101bbf45654628b9f2d780bc13fcbf'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:46.054860 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-11d8b0e9-4329-4b88-ab43-eb37a733cf77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d101bbf45654628b9f2d780bc13fcbf'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:46.162100 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd57a274-4ece-4c32-8da7-70ff808ce37f service neutron] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:46.164279 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-dd57a274-4ece-4c32-8da7-70ff808ce37f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:46.204623 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1013/2054] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:28:45 2022] DELETE /identity/v3/users/2d101bbf45654628b9f2d780bc13fcbf => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:28:46.209808 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3e3a51-f9a0-4c56-8178-36c38e253b40 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:46.210632 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3e3a51-f9a0-4c56-8178-36c38e253b40 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:46.211261 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3e3a51-f9a0-4c56-8178-36c38e253b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:46.222126 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd57a274-4ece-4c32-8da7-70ff808ce37f service neutron] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:46.233955 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-8f3e3a51-f9a0-4c56-8178-36c38e253b40 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:46.235262 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-8f3e3a51-f9a0-4c56-8178-36c38e253b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:46.316184 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1014/2055] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:28:46.321278 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6be777-d528-4b44-ad4d-c37cab0be52b None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:46.322156 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6be777-d528-4b44-ad4d-c37cab0be52b None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:46.322545 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6be777-d528-4b44-ad4d-c37cab0be52b None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:46.324411 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1015/2056] 10.222.0.23 () {54 vars in 921 bytes} [Tue Jun 21 18:28:46 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:28:46.327637 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6be777-d528-4b44-ad4d-c37cab0be52b None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:46.331189 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-3d6be777-d528-4b44-ad4d-c37cab0be52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:46.337890 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3cf29f-c129-4716-973b-571af247ac58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:46.339740 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3cf29f-c129-4716-973b-571af247ac58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:46.340137 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3cf29f-c129-4716-973b-571af247ac58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:46.340446 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3cf29f-c129-4716-973b-571af247ac58 admin admin] PATH_INFO: `/v3/projects/250848114c81496bab02299e94a26f02` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:46.341728 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3cf29f-c129-4716-973b-571af247ac58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=250848114c81496bab02299e94a26f02)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:46.343689 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3cf29f-c129-4716-973b-571af247ac58 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:46.416430 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6be777-d528-4b44-ad4d-c37cab0be52b None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:46.435801 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44ccd07-df1a-4bdd-be08-ada776789077 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:46.436511 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ccd07-df1a-4bdd-be08-ada776789077 service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:46.436796 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ccd07-df1a-4bdd-be08-ada776789077 service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:46.437070 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ccd07-df1a-4bdd-be08-ada776789077 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:46.451344 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44ccd07-df1a-4bdd-be08-ada776789077 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:46.453198 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44ccd07-df1a-4bdd-be08-ada776789077 service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:46.465865 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1016/2057] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:28:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:28:46.467347 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d0c92-79fb-446f-ace2-f147fe410b94 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:46.467558 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d0c92-79fb-446f-ace2-f147fe410b94 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:46.467840 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d0c92-79fb-446f-ace2-f147fe410b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:46.501322 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-be0d0c92-79fb-446f-ace2-f147fe410b94 None None] MFA Rules not processed for user `938187967efe49be9c5389d1feef81ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:46.502611 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-be0d0c92-79fb-446f-ace2-f147fe410b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:46.521855 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:46.543757 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-ce3cf29f-c129-4716-973b-571af247ac58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '250848114c81496bab02299e94a26f02', 'request_id': 'req-ce3cf29f-c129-4716-973b-571af247ac58'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:46.550694 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-ce3cf29f-c129-4716-973b-571af247ac58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '250848114c81496bab02299e94a26f02', 'request_id': 'req-ce3cf29f-c129-4716-973b-571af247ac58'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:46.557174 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-ce3cf29f-c129-4716-973b-571af247ac58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '250848114c81496bab02299e94a26f02', 'request_id': 'req-ce3cf29f-c129-4716-973b-571af247ac58'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:46.583600 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1017/2058] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Jun 21 18:28:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:28:46.584503 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0d0c92-79fb-446f-ace2-f147fe410b94 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:46.585776 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-be0d0c92-79fb-446f-ace2-f147fe410b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:46.600080 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0d0c92-79fb-446f-ace2-f147fe410b94 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:46.656946 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddca98d3-1269-452a-bddb-663987f411c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:46.657825 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddca98d3-1269-452a-bddb-663987f411c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:46.658103 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddca98d3-1269-452a-bddb-663987f411c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:46.658372 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddca98d3-1269-452a-bddb-663987f411c1 admin admin] PATH_INFO: `/v3/users/35d70344216248199ddf81fba88af04f` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:46.659852 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddca98d3-1269-452a-bddb-663987f411c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35d70344216248199ddf81fba88af04f)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:46.661614 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddca98d3-1269-452a-bddb-663987f411c1 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:46.708801 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-ce3cf29f-c129-4716-973b-571af247ac58 admin admin] The token cache is being invalidate because project 250848114c81496bab02299e94a26f02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:28:46.709100 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-ce3cf29f-c129-4716-973b-571af247ac58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:46.709759 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-ce3cf29f-c129-4716-973b-571af247ac58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:46.711511 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1042/2059] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:28:46 2022] DELETE /identity/v3/projects/250848114c81496bab02299e94a26f02 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:28:46.712572 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3cf29f-c129-4716-973b-571af247ac58 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:46.715692 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-ce3cf29f-c129-4716-973b-571af247ac58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:46.793511 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3cf29f-c129-4716-973b-571af247ac58 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:46.799812 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c30ae4-f420-4904-9195-814d999bea68 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938187967efe49be9c5389d1feef81ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:46.800522 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c30ae4-f420-4904-9195-814d999bea68 service placement] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:46.800857 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c30ae4-f420-4904-9195-814d999bea68 service placement] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:46.801113 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c30ae4-f420-4904-9195-814d999bea68 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:46.802967 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-c6c30ae4-f420-4904-9195-814d999bea68 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:46.816106 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c30ae4-f420-4904-9195-814d999bea68 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:46.817741 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c30ae4-f420-4904-9195-814d999bea68 service placement] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:46.826522 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:46.828997 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1043/2060] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:28:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:28:46.829909 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c30ae4-f420-4904-9195-814d999bea68 service placement] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:46.831382 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-c6c30ae4-f420-4904-9195-814d999bea68 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:46.845004 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c30ae4-f420-4904-9195-814d999bea68 service placement] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:46.849958 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681c016e-5d58-4ecc-bef5-d2019cbfba02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:46.855216 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681c016e-5d58-4ecc-bef5-d2019cbfba02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:46.855506 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681c016e-5d58-4ecc-bef5-d2019cbfba02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:46.855829 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681c016e-5d58-4ecc-bef5-d2019cbfba02 admin admin] PATH_INFO: `/v3/users/88452e8587964fae866fb6014424a170` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:46.866718 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681c016e-5d58-4ecc-bef5-d2019cbfba02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88452e8587964fae866fb6014424a170)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:46.868796 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681c016e-5d58-4ecc-bef5-d2019cbfba02 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:46.918050 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-ddca98d3-1269-452a-bddb-663987f411c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35d70344216248199ddf81fba88af04f'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:46.931883 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-ddca98d3-1269-452a-bddb-663987f411c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35d70344216248199ddf81fba88af04f'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:46.933837 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-ddca98d3-1269-452a-bddb-663987f411c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35d70344216248199ddf81fba88af04f'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:46.940891 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-ddca98d3-1269-452a-bddb-663987f411c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35d70344216248199ddf81fba88af04f'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:46.941960 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-ddca98d3-1269-452a-bddb-663987f411c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35d70344216248199ddf81fba88af04f'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:47.052189 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1018/2061] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:28:46 2022] DELETE /identity/v3/users/35d70344216248199ddf81fba88af04f => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:47.053032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddca98d3-1269-452a-bddb-663987f411c1 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:47.054410 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-ddca98d3-1269-452a-bddb-663987f411c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:47.097776 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-681c016e-5d58-4ecc-bef5-d2019cbfba02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88452e8587964fae866fb6014424a170'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:47.116928 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-681c016e-5d58-4ecc-bef5-d2019cbfba02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88452e8587964fae866fb6014424a170'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:47.125893 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddca98d3-1269-452a-bddb-663987f411c1 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:47.130738 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1d5d7-5ee4-4148-836d-b875b5f5f299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:47.131627 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1d5d7-5ee4-4148-836d-b875b5f5f299 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:47.131916 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1d5d7-5ee4-4148-836d-b875b5f5f299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:47.132194 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1d5d7-5ee4-4148-836d-b875b5f5f299 admin admin] PATH_INFO: `/v3/projects/20eedaa8dc5e474fa7b74bd4b75b95b2` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:47.133432 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec1d5d7-5ee4-4148-836d-b875b5f5f299 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20eedaa8dc5e474fa7b74bd4b75b95b2)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:47.136240 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec1d5d7-5ee4-4148-836d-b875b5f5f299 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:47.160139 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-681c016e-5d58-4ecc-bef5-d2019cbfba02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88452e8587964fae866fb6014424a170'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:47.160863 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-681c016e-5d58-4ecc-bef5-d2019cbfba02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88452e8587964fae866fb6014424a170'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:47.169721 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-681c016e-5d58-4ecc-bef5-d2019cbfba02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88452e8587964fae866fb6014424a170'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:47.176050 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1044/2062] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:28:46 2022] DELETE /identity/v3/users/88452e8587964fae866fb6014424a170 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:47.261920 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-7ec1d5d7-5ee4-4148-836d-b875b5f5f299 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20eedaa8dc5e474fa7b74bd4b75b95b2', 'request_id': 'req-7ec1d5d7-5ee4-4148-836d-b875b5f5f299'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:47.333239 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-7ec1d5d7-5ee4-4148-836d-b875b5f5f299 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20eedaa8dc5e474fa7b74bd4b75b95b2', 'request_id': 'req-7ec1d5d7-5ee4-4148-836d-b875b5f5f299'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:47.346810 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681c016e-5d58-4ecc-bef5-d2019cbfba02 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:47.348571 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-681c016e-5d58-4ecc-bef5-d2019cbfba02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:47.349167 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-7ec1d5d7-5ee4-4148-836d-b875b5f5f299 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20eedaa8dc5e474fa7b74bd4b75b95b2', 'request_id': 'req-7ec1d5d7-5ee4-4148-836d-b875b5f5f299'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:47.363430 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-7ec1d5d7-5ee4-4148-836d-b875b5f5f299 admin admin] The token cache is being invalidate because project 20eedaa8dc5e474fa7b74bd4b75b95b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:28:47.363734 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-7ec1d5d7-5ee4-4148-836d-b875b5f5f299 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:47.365272 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-7ec1d5d7-5ee4-4148-836d-b875b5f5f299 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:47.367234 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1019/2063] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:28:47 2022] DELETE /identity/v3/projects/20eedaa8dc5e474fa7b74bd4b75b95b2 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:47.384634 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1d5d7-5ee4-4148-836d-b875b5f5f299 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:47.388565 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-7ec1d5d7-5ee4-4148-836d-b875b5f5f299 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:47.500568 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1d5d7-5ee4-4148-836d-b875b5f5f299 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:47.506729 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64589e2d-c457-4068-ad6b-c7b1a8a468e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:47.507645 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64589e2d-c457-4068-ad6b-c7b1a8a468e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:47.508020 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64589e2d-c457-4068-ad6b-c7b1a8a468e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:47.508198 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64589e2d-c457-4068-ad6b-c7b1a8a468e6 admin admin] PATH_INFO: `/v3/users/f32f185c3afd45fc8295987622b6d17f` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:47.509482 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64589e2d-c457-4068-ad6b-c7b1a8a468e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f32f185c3afd45fc8295987622b6d17f)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:47.512906 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64589e2d-c457-4068-ad6b-c7b1a8a468e6 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:47.525966 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681c016e-5d58-4ecc-bef5-d2019cbfba02 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:47.549330 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57486b0-348a-4f64-8f36-c1af89a25f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:47.550509 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57486b0-348a-4f64-8f36-c1af89a25f55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:47.551042 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57486b0-348a-4f64-8f36-c1af89a25f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:47.551393 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57486b0-348a-4f64-8f36-c1af89a25f55 admin admin] PATH_INFO: `/v3/projects/7e5e18f0b6ab45ef961c14b967941ca6` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:47.553127 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57486b0-348a-4f64-8f36-c1af89a25f55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e5e18f0b6ab45ef961c14b967941ca6)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:47.555431 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57486b0-348a-4f64-8f36-c1af89a25f55 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:47.601644 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:47.676464 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-64589e2d-c457-4068-ad6b-c7b1a8a468e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f32f185c3afd45fc8295987622b6d17f'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:47.689250 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-64589e2d-c457-4068-ad6b-c7b1a8a468e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f32f185c3afd45fc8295987622b6d17f'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:47.690096 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-64589e2d-c457-4068-ad6b-c7b1a8a468e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f32f185c3afd45fc8295987622b6d17f'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:47.696850 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-64589e2d-c457-4068-ad6b-c7b1a8a468e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f32f185c3afd45fc8295987622b6d17f'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:47.697621 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-64589e2d-c457-4068-ad6b-c7b1a8a468e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f32f185c3afd45fc8295987622b6d17f'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:47.742979 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-b57486b0-348a-4f64-8f36-c1af89a25f55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e5e18f0b6ab45ef961c14b967941ca6', 'request_id': 'req-b57486b0-348a-4f64-8f36-c1af89a25f55'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:47.750640 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-b57486b0-348a-4f64-8f36-c1af89a25f55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e5e18f0b6ab45ef961c14b967941ca6', 'request_id': 'req-b57486b0-348a-4f64-8f36-c1af89a25f55'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:47.756756 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-b57486b0-348a-4f64-8f36-c1af89a25f55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e5e18f0b6ab45ef961c14b967941ca6', 'request_id': 'req-b57486b0-348a-4f64-8f36-c1af89a25f55'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:47.766470 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:47.790523 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1020/2064] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:28:47 2022] DELETE /identity/v3/users/f32f185c3afd45fc8295987622b6d17f => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:47.794982 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64589e2d-c457-4068-ad6b-c7b1a8a468e6 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:47.796377 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-64589e2d-c457-4068-ad6b-c7b1a8a468e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:47.839829 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-b57486b0-348a-4f64-8f36-c1af89a25f55 admin admin] The token cache is being invalidate because project 7e5e18f0b6ab45ef961c14b967941ca6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:28:47.840273 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-b57486b0-348a-4f64-8f36-c1af89a25f55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:47.841150 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-b57486b0-348a-4f64-8f36-c1af89a25f55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:47.848370 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1045/2065] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:28:47 2022] DELETE /identity/v3/projects/7e5e18f0b6ab45ef961c14b967941ca6 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:47.850307 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdf1189-265e-4478-880f-5608c261131d None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:47.850690 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdf1189-265e-4478-880f-5608c261131d None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:47.851053 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdf1189-265e-4478-880f-5608c261131d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:47.889629 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-7bdf1189-265e-4478-880f-5608c261131d None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:47.892376 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-7bdf1189-265e-4478-880f-5608c261131d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:47.908296 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64589e2d-c457-4068-ad6b-c7b1a8a468e6 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:47.918372 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88e8b37-ccc3-4bd9-8a36-97afadf6d8f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:47.918971 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e8b37-ccc3-4bd9-8a36-97afadf6d8f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:47.919256 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e8b37-ccc3-4bd9-8a36-97afadf6d8f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:47.919538 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e8b37-ccc3-4bd9-8a36-97afadf6d8f2 admin admin] PATH_INFO: `/v3/projects/7a96532be6ee4f1ab16e6e3024fa1e9f` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:47.920816 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88e8b37-ccc3-4bd9-8a36-97afadf6d8f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a96532be6ee4f1ab16e6e3024fa1e9f)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:47.923120 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88e8b37-ccc3-4bd9-8a36-97afadf6d8f2 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:47.962406 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1046/2066] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:28:47.971082 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e5dacd-1c69-4966-a805-c7297dbbdb4b None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:47.972442 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e5dacd-1c69-4966-a805-c7297dbbdb4b None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:47.972442 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e5dacd-1c69-4966-a805-c7297dbbdb4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:47.990623 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-f7e5dacd-1c69-4966-a805-c7297dbbdb4b None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:47.991842 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-f7e5dacd-1c69-4966-a805-c7297dbbdb4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:48.000527 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1047/2067] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:28:48.005205 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e5dacd-1c69-4966-a805-c7297dbbdb4b None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:48.015835 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e5dacd-1c69-4966-a805-c7297dbbdb4b None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:48.020690 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c6f335-af94-419e-b6b4-477459cbfdd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:48.021361 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c6f335-af94-419e-b6b4-477459cbfdd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:48.021647 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c6f335-af94-419e-b6b4-477459cbfdd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:48.027398 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c6f335-af94-419e-b6b4-477459cbfdd7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:48.027398 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c6f335-af94-419e-b6b4-477459cbfdd7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:28:48.027398 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c6f335-af94-419e-b6b4-477459cbfdd7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:48.027398 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c6f335-af94-419e-b6b4-477459cbfdd7 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:48.044228 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1048/2068] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:28:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:28:48.045451 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c6f335-af94-419e-b6b4-477459cbfdd7 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:48.052923 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-b88e8b37-ccc3-4bd9-8a36-97afadf6d8f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a96532be6ee4f1ab16e6e3024fa1e9f', 'request_id': 'req-b88e8b37-ccc3-4bd9-8a36-97afadf6d8f2'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:48.055575 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c6f335-af94-419e-b6b4-477459cbfdd7 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:48.116201 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbdc317-f67d-45bd-9247-e70e39ab6b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:48.116817 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbdc317-f67d-45bd-9247-e70e39ab6b48 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:48.117102 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbdc317-f67d-45bd-9247-e70e39ab6b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:48.117379 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbdc317-f67d-45bd-9247-e70e39ab6b48 admin admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:48.118075 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbdc317-f67d-45bd-9247-e70e39ab6b48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:48.120117 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbdc317-f67d-45bd-9247-e70e39ab6b48 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:48.133053 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-b88e8b37-ccc3-4bd9-8a36-97afadf6d8f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a96532be6ee4f1ab16e6e3024fa1e9f', 'request_id': 'req-b88e8b37-ccc3-4bd9-8a36-97afadf6d8f2'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:48.141524 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-b88e8b37-ccc3-4bd9-8a36-97afadf6d8f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a96532be6ee4f1ab16e6e3024fa1e9f', 'request_id': 'req-b88e8b37-ccc3-4bd9-8a36-97afadf6d8f2'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:48.154344 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-b88e8b37-ccc3-4bd9-8a36-97afadf6d8f2 admin admin] The token cache is being invalidate because project 7a96532be6ee4f1ab16e6e3024fa1e9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:28:48.154622 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-b88e8b37-ccc3-4bd9-8a36-97afadf6d8f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:48.155292 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-b88e8b37-ccc3-4bd9-8a36-97afadf6d8f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:48.156941 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1021/2069] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:28:47 2022] DELETE /identity/v3/projects/7a96532be6ee4f1ab16e6e3024fa1e9f => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:48.161564 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28680daf-0fae-4bc2-93c8-3a1f0c91342c None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:48.161854 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28680daf-0fae-4bc2-93c8-3a1f0c91342c None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:48.168185 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28680daf-0fae-4bc2-93c8-3a1f0c91342c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:48.181923 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-28680daf-0fae-4bc2-93c8-3a1f0c91342c None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:48.184443 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-28680daf-0fae-4bc2-93c8-3a1f0c91342c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:48.198945 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1022/2070] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:28:48.200866 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233cd893-e6be-4493-b904-2f1932b58cab None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:48.201157 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233cd893-e6be-4493-b904-2f1932b58cab None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:48.201436 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233cd893-e6be-4493-b904-2f1932b58cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:48.219984 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-233cd893-e6be-4493-b904-2f1932b58cab None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:48.221206 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-233cd893-e6be-4493-b904-2f1932b58cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:48.225727 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1049/2071] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:28:48 2022] POST /identity/v3/projects => generated 379 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:28:48.232233 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbdc317-f67d-45bd-9247-e70e39ab6b48 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:48.234134 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-8dbdc317-f67d-45bd-9247-e70e39ab6b48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:48.285777 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1023/2072] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:28:48.290007 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233cd893-e6be-4493-b904-2f1932b58cab None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:48.300770 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233cd893-e6be-4493-b904-2f1932b58cab None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:48.300982 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbdc317-f67d-45bd-9247-e70e39ab6b48 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:48.306134 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c26491-c074-40ed-a08c-502a061512ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:48.306267 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00daa12a-f466-4447-a953-be49f576b1f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:48.309171 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00daa12a-f466-4447-a953-be49f576b1f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:48.313861 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c26491-c074-40ed-a08c-502a061512ac admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:48.313969 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00daa12a-f466-4447-a953-be49f576b1f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:48.314078 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c26491-c074-40ed-a08c-502a061512ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:48.314166 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00daa12a-f466-4447-a953-be49f576b1f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:48.314269 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c26491-c074-40ed-a08c-502a061512ac admin admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:48.314365 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00daa12a-f466-4447-a953-be49f576b1f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:28:48.314466 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c26491-c074-40ed-a08c-502a061512ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:48.314553 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00daa12a-f466-4447-a953-be49f576b1f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:48.314553 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00daa12a-f466-4447-a953-be49f576b1f1 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:48.314745 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c26491-c074-40ed-a08c-502a061512ac admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:48.327528 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1024/2073] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:28:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:28:48.327528 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00daa12a-f466-4447-a953-be49f576b1f1 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:48.328842 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-00daa12a-f466-4447-a953-be49f576b1f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:48.343470 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00daa12a-f466-4447-a953-be49f576b1f1 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:48.348309 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce18a8bf-de06-4505-b6ca-5fbfdd5b85d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:48.349219 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce18a8bf-de06-4505-b6ca-5fbfdd5b85d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:48.349500 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce18a8bf-de06-4505-b6ca-5fbfdd5b85d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:48.349775 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce18a8bf-de06-4505-b6ca-5fbfdd5b85d8 admin admin] PATH_INFO: `/v3/projects/583b69cd118e4998b2f23331eff9b9ac` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:48.351007 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce18a8bf-de06-4505-b6ca-5fbfdd5b85d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=583b69cd118e4998b2f23331eff9b9ac)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:48.353056 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce18a8bf-de06-4505-b6ca-5fbfdd5b85d8 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:48.376046 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1050/2074] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:28:48 2022] POST /identity/v3/users => generated 290 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:28:48.376046 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c26491-c074-40ed-a08c-502a061512ac admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:48.384476 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c26491-c074-40ed-a08c-502a061512ac admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:48.389397 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f96b10b-73f2-4fdd-b4bc-7629df12541e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:48.390056 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f96b10b-73f2-4fdd-b4bc-7629df12541e admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:48.390337 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f96b10b-73f2-4fdd-b4bc-7629df12541e admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:48.390612 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f96b10b-73f2-4fdd-b4bc-7629df12541e admin admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:48.391378 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f96b10b-73f2-4fdd-b4bc-7629df12541e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:48.393372 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f96b10b-73f2-4fdd-b4bc-7629df12541e admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:48.492463 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-ce18a8bf-de06-4505-b6ca-5fbfdd5b85d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '583b69cd118e4998b2f23331eff9b9ac', 'request_id': 'req-ce18a8bf-de06-4505-b6ca-5fbfdd5b85d8'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:48.507967 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1051/2075] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:28:48 2022] POST /identity/v3/projects => generated 399 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:28:48.507967 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f96b10b-73f2-4fdd-b4bc-7629df12541e admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:48.522393 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:48.577725 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-ce18a8bf-de06-4505-b6ca-5fbfdd5b85d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '583b69cd118e4998b2f23331eff9b9ac', 'request_id': 'req-ce18a8bf-de06-4505-b6ca-5fbfdd5b85d8'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:48.585022 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f96b10b-73f2-4fdd-b4bc-7629df12541e admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:48.586502 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-ce18a8bf-de06-4505-b6ca-5fbfdd5b85d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '583b69cd118e4998b2f23331eff9b9ac', 'request_id': 'req-ce18a8bf-de06-4505-b6ca-5fbfdd5b85d8'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:48.591742 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b0ecaa-4c6b-4079-8483-72a7eed99a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:48.592483 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b0ecaa-4c6b-4079-8483-72a7eed99a8a admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:48.592876 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b0ecaa-4c6b-4079-8483-72a7eed99a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:48.593144 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b0ecaa-4c6b-4079-8483-72a7eed99a8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:48.594083 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b0ecaa-4c6b-4079-8483-72a7eed99a8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:48.595732 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b0ecaa-4c6b-4079-8483-72a7eed99a8a admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:48.604135 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:48.607461 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1052/2076] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:28:48 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:28:48.608422 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b0ecaa-4c6b-4079-8483-72a7eed99a8a admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:48.610259 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-ce18a8bf-de06-4505-b6ca-5fbfdd5b85d8 admin admin] The token cache is being invalidate because project 583b69cd118e4998b2f23331eff9b9ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:28:48.610624 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-ce18a8bf-de06-4505-b6ca-5fbfdd5b85d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:48.611484 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-ce18a8bf-de06-4505-b6ca-5fbfdd5b85d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:48.613117 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1025/2077] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:28:48 2022] DELETE /identity/v3/projects/583b69cd118e4998b2f23331eff9b9ac => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:48.618155 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce18a8bf-de06-4505-b6ca-5fbfdd5b85d8 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:48.619963 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-ce18a8bf-de06-4505-b6ca-5fbfdd5b85d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:48.621988 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b0ecaa-4c6b-4079-8483-72a7eed99a8a admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:48.629510 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11419678-abdb-4c85-bf31-6c8e0534d3da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:48.630650 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11419678-abdb-4c85-bf31-6c8e0534d3da admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:48.631087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11419678-abdb-4c85-bf31-6c8e0534d3da admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:48.631599 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11419678-abdb-4c85-bf31-6c8e0534d3da admin admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:48.632802 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11419678-abdb-4c85-bf31-6c8e0534d3da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:48.635916 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11419678-abdb-4c85-bf31-6c8e0534d3da admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:48.696172 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce18a8bf-de06-4505-b6ca-5fbfdd5b85d8 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:48.701593 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cdb166-a0f2-4ea4-a29c-6121407a0c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:48.702292 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cdb166-a0f2-4ea4-a29c-6121407a0c55 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:48.702581 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cdb166-a0f2-4ea4-a29c-6121407a0c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:48.702860 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cdb166-a0f2-4ea4-a29c-6121407a0c55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:48.703608 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cdb166-a0f2-4ea4-a29c-6121407a0c55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:48.707552 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cdb166-a0f2-4ea4-a29c-6121407a0c55 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:48.721571 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1026/2078] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:28:48 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:28:48.721571 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c35705-9767-4b17-990d-5b1472c53072 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:48.721571 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c35705-9767-4b17-990d-5b1472c53072 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:48.721571 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c35705-9767-4b17-990d-5b1472c53072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:48.740472 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-b0c35705-9767-4b17-990d-5b1472c53072 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:48.741727 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-b0c35705-9767-4b17-990d-5b1472c53072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:48.750860 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1027/2079] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:28:48.751488 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c35705-9767-4b17-990d-5b1472c53072 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:48.762025 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c35705-9767-4b17-990d-5b1472c53072 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:48.768614 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddd54dd-8241-4b77-b54b-05aef7cfdcf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:48.769839 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd54dd-8241-4b77-b54b-05aef7cfdcf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:48.770236 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd54dd-8241-4b77-b54b-05aef7cfdcf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:48.776775 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd54dd-8241-4b77-b54b-05aef7cfdcf9 admin admin] PATH_INFO: `/v3/projects/23d613c758b94f498ff7d0d996b7b7db/users/0259af286ce24d928a13b2aeb0700246/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:48.776910 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1053/2080] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:28:48 2022] POST /identity/v3/users => generated 300 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:28:48.776910 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c01501-6c84-4612-ae3e-54fe9f03eb43 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:48.776910 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c01501-6c84-4612-ae3e-54fe9f03eb43 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:48.776910 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c01501-6c84-4612-ae3e-54fe9f03eb43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:48.792503 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddd54dd-8241-4b77-b54b-05aef7cfdcf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23d613c758b94f498ff7d0d996b7b7db, user_id=0259af286ce24d928a13b2aeb0700246, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:48.795324 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddd54dd-8241-4b77-b54b-05aef7cfdcf9 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:48.805033 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-f6c01501-6c84-4612-ae3e-54fe9f03eb43 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:48.806345 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-f6c01501-6c84-4612-ae3e-54fe9f03eb43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:48.821779 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1054/2081] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:28:48.821779 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c01501-6c84-4612-ae3e-54fe9f03eb43 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:48.822092 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-f6c01501-6c84-4612-ae3e-54fe9f03eb43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:48.834970 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c01501-6c84-4612-ae3e-54fe9f03eb43 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:48.840607 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20b8234-061a-4b47-8e22-7e0b79816e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:48.841320 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20b8234-061a-4b47-8e22-7e0b79816e6c admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:48.841609 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20b8234-061a-4b47-8e22-7e0b79816e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:48.841884 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20b8234-061a-4b47-8e22-7e0b79816e6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:48.842642 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20b8234-061a-4b47-8e22-7e0b79816e6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:48.844343 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20b8234-061a-4b47-8e22-7e0b79816e6c admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:48.858000 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1055/2082] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:28:48 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:28:48.858000 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20b8234-061a-4b47-8e22-7e0b79816e6c admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:48.866421 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20b8234-061a-4b47-8e22-7e0b79816e6c admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:48.875115 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b1835c-f1ac-4bab-8630-42ba52692653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:48.879461 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b1835c-f1ac-4bab-8630-42ba52692653 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:48.879461 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b1835c-f1ac-4bab-8630-42ba52692653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:48.879461 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b1835c-f1ac-4bab-8630-42ba52692653 admin admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:48.879461 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b1835c-f1ac-4bab-8630-42ba52692653 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:28:48.879461 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b1835c-f1ac-4bab-8630-42ba52692653 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:48.879993 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1028/2083] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:28:48 2022] PUT /identity/v3/projects/23d613c758b94f498ff7d0d996b7b7db/users/0259af286ce24d928a13b2aeb0700246/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:48.879993 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddd54dd-8241-4b77-b54b-05aef7cfdcf9 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:48.881086 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b1835c-f1ac-4bab-8630-42ba52692653 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:48.896493 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1056/2084] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:28:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:28:48.901894 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bebd94b-86a5-4e48-a10d-bb41032354bc None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:48.910716 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bebd94b-86a5-4e48-a10d-bb41032354bc None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:48.911280 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bebd94b-86a5-4e48-a10d-bb41032354bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:48.953835 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-4bebd94b-86a5-4e48-a10d-bb41032354bc None None] MFA Rules not processed for user `0259af286ce24d928a13b2aeb0700246`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:48.959338 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-4bebd94b-86a5-4e48-a10d-bb41032354bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:49.029460 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddd54dd-8241-4b77-b54b-05aef7cfdcf9 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:49.044091 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1057/2085] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:48 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:28:49.044835 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5cfc3e-38f8-4256-a3d7-cb640868f747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:49.045086 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bebd94b-86a5-4e48-a10d-bb41032354bc None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:49.046118 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5cfc3e-38f8-4256-a3d7-cb640868f747 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:49.046509 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5cfc3e-38f8-4256-a3d7-cb640868f747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:49.046688 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5cfc3e-38f8-4256-a3d7-cb640868f747 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:49.047721 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5cfc3e-38f8-4256-a3d7-cb640868f747 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:49.049433 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5cfc3e-38f8-4256-a3d7-cb640868f747 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:49.058360 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bebd94b-86a5-4e48-a10d-bb41032354bc None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:49.062300 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1029/2086] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:28:48 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:28:49.063157 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5cfc3e-38f8-4256-a3d7-cb640868f747 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:49.068270 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c739ab42-51d1-4c9e-99cf-bc472f2e5dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:49.069238 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c739ab42-51d1-4c9e-99cf-bc472f2e5dce admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:49.069688 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c739ab42-51d1-4c9e-99cf-bc472f2e5dce admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:49.070113 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c739ab42-51d1-4c9e-99cf-bc472f2e5dce admin admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:49.071002 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c739ab42-51d1-4c9e-99cf-bc472f2e5dce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:49.078742 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c739ab42-51d1-4c9e-99cf-bc472f2e5dce admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:49.096709 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5cfc3e-38f8-4256-a3d7-cb640868f747 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:49.096709 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55486044-a1c6-44d8-9e21-72bb321c35b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:49.096709 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55486044-a1c6-44d8-9e21-72bb321c35b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:49.096709 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55486044-a1c6-44d8-9e21-72bb321c35b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:49.097500 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55486044-a1c6-44d8-9e21-72bb321c35b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:49.097500 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55486044-a1c6-44d8-9e21-72bb321c35b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:49.097500 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55486044-a1c6-44d8-9e21-72bb321c35b4 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:49.150598 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:49.386466 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:49.393295 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1058/2087] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:28:49 2022] POST /identity/v3/projects => generated 375 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:28:49.393974 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c739ab42-51d1-4c9e-99cf-bc472f2e5dce admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:49.394595 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1030/2088] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:28:49 2022] POST /identity/v3/projects => generated 379 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:28:49.397532 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55486044-a1c6-44d8-9e21-72bb321c35b4 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:49.483971 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c739ab42-51d1-4c9e-99cf-bc472f2e5dce admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:49.486581 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55486044-a1c6-44d8-9e21-72bb321c35b4 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:49.490451 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f25728-cde8-4fea-b13a-f8a0d7d6dd82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:49.492100 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f25728-cde8-4fea-b13a-f8a0d7d6dd82 admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:49.492487 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f25728-cde8-4fea-b13a-f8a0d7d6dd82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:49.492764 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f25728-cde8-4fea-b13a-f8a0d7d6dd82 admin admin] PATH_INFO: `/v3/projects/828af27280204701b39fa3ed06fc9bdc/users/1b67f011b29c4985bdbb5c212babc43a/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:49.493209 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e0f7c3-bd7a-478d-8137-1db909534ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:49.493910 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e0f7c3-bd7a-478d-8137-1db909534ea9 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:49.494250 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e0f7c3-bd7a-478d-8137-1db909534ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:49.494537 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e0f7c3-bd7a-478d-8137-1db909534ea9 admin admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:49.496559 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e0f7c3-bd7a-478d-8137-1db909534ea9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:49.501222 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e0f7c3-bd7a-478d-8137-1db909534ea9 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:49.514718 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f25728-cde8-4fea-b13a-f8a0d7d6dd82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=828af27280204701b39fa3ed06fc9bdc, user_id=1b67f011b29c4985bdbb5c212babc43a, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:49.519428 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f25728-cde8-4fea-b13a-f8a0d7d6dd82 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:49.619146 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1059/2089] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:28:49 2022] PUT /identity/v3/projects/828af27280204701b39fa3ed06fc9bdc/users/1b67f011b29c4985bdbb5c212babc43a/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:49.621261 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f25728-cde8-4fea-b13a-f8a0d7d6dd82 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:49.632496 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1031/2090] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:28:49 2022] POST /identity/v3/users => generated 288 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:28:49.633558 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e0f7c3-bd7a-478d-8137-1db909534ea9 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:49.643983 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e0f7c3-bd7a-478d-8137-1db909534ea9 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:49.696235 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f25728-cde8-4fea-b13a-f8a0d7d6dd82 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:49.702304 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d657ad54-2b83-4863-aeea-80ea5576db38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:49.703018 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d657ad54-2b83-4863-aeea-80ea5576db38 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:49.703318 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d657ad54-2b83-4863-aeea-80ea5576db38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:49.703580 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d657ad54-2b83-4863-aeea-80ea5576db38 admin admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:49.704295 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d657ad54-2b83-4863-aeea-80ea5576db38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:49.706376 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d657ad54-2b83-4863-aeea-80ea5576db38 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:49.716056 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0fdb1e-70de-4ee9-97e2-c08280f9f27c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:49.716717 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0fdb1e-70de-4ee9-97e2-c08280f9f27c admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:49.717214 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0fdb1e-70de-4ee9-97e2-c08280f9f27c admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:49.717335 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0fdb1e-70de-4ee9-97e2-c08280f9f27c admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:49.718052 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0fdb1e-70de-4ee9-97e2-c08280f9f27c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:49.719885 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0fdb1e-70de-4ee9-97e2-c08280f9f27c admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:49.731525 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1032/2091] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:28:49.733492 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bcd60a-820a-4325-bbc6-e6ad7906ea6e None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:49.733849 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bcd60a-820a-4325-bbc6-e6ad7906ea6e None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:49.734193 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bcd60a-820a-4325-bbc6-e6ad7906ea6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:49.765065 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-00bcd60a-820a-4325-bbc6-e6ad7906ea6e None None] MFA Rules not processed for user `1b67f011b29c4985bdbb5c212babc43a`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:49.766312 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-00bcd60a-820a-4325-bbc6-e6ad7906ea6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:49.824428 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1060/2092] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:28:49 2022] POST /identity/v3/users => generated 296 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:28:49.825482 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d657ad54-2b83-4863-aeea-80ea5576db38 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:49.838122 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d657ad54-2b83-4863-aeea-80ea5576db38 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:49.845461 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250b025a-6851-403e-b888-4e1e43321a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:49.846540 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b025a-6851-403e-b888-4e1e43321a80 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:49.848319 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b025a-6851-403e-b888-4e1e43321a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:49.848319 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b025a-6851-403e-b888-4e1e43321a80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:49.855407 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250b025a-6851-403e-b888-4e1e43321a80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:49.855407 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250b025a-6851-403e-b888-4e1e43321a80 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:49.870337 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1033/2093] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:49 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:28:49.872338 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1061/2094] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:28:49.876215 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250b025a-6851-403e-b888-4e1e43321a80 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:49.876427 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bcd60a-820a-4325-bbc6-e6ad7906ea6e None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:49.887642 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250b025a-6851-403e-b888-4e1e43321a80 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:49.890245 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bcd60a-820a-4325-bbc6-e6ad7906ea6e None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:49.895246 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c40f2-d179-4286-a3c2-7fbfb0cffbea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:49.895828 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012fd314-6955-4393-b830-f549c32f3e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:49.896604 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c40f2-d179-4286-a3c2-7fbfb0cffbea admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:49.896739 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012fd314-6955-4393-b830-f549c32f3e3b admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:49.896903 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012fd314-6955-4393-b830-f549c32f3e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:49.897180 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012fd314-6955-4393-b830-f549c32f3e3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:49.897320 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c40f2-d179-4286-a3c2-7fbfb0cffbea admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:49.897485 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c40f2-d179-4286-a3c2-7fbfb0cffbea admin admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:49.897978 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012fd314-6955-4393-b830-f549c32f3e3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:49.898215 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9c40f2-d179-4286-a3c2-7fbfb0cffbea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:49.899597 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012fd314-6955-4393-b830-f549c32f3e3b admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:49.900190 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9c40f2-d179-4286-a3c2-7fbfb0cffbea admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:49.917931 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1034/2095] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:28:49.919349 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012fd314-6955-4393-b830-f549c32f3e3b admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:49.937551 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012fd314-6955-4393-b830-f549c32f3e3b admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:49.943618 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:49.945578 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c476e5fc-1b80-4327-897b-ce67565d0a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:49.946536 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476e5fc-1b80-4327-897b-ce67565d0a5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:49.947231 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476e5fc-1b80-4327-897b-ce67565d0a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:49.947231 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476e5fc-1b80-4327-897b-ce67565d0a5b admin admin] PATH_INFO: `/v3/projects/c30e59eecc944c30837fc6164c4a89ea/users/33737a66aafb4884b23f9680a8d4060f/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:49.965832 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c476e5fc-1b80-4327-897b-ce67565d0a5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c30e59eecc944c30837fc6164c4a89ea, user_id=33737a66aafb4884b23f9680a8d4060f, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:49.968618 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c476e5fc-1b80-4327-897b-ce67565d0a5b admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:49.997553 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1062/2096] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:28:49 2022] POST /identity/v3/projects => generated 397 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:28:49.999657 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c40f2-d179-4286-a3c2-7fbfb0cffbea admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:50.082640 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c40f2-d179-4286-a3c2-7fbfb0cffbea admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:50.087563 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54717840-d92b-40ba-8124-ad0a6cbb63ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:50.089103 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54717840-d92b-40ba-8124-ad0a6cbb63ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:50.089400 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54717840-d92b-40ba-8124-ad0a6cbb63ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:50.089847 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54717840-d92b-40ba-8124-ad0a6cbb63ce admin admin] PATH_INFO: `/v3/projects/10e761f41b8949e787f4fb828407283e/users/bdc73f789dc944638aaaea07234c5dda/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:50.105009 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1035/2097] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:28:49 2022] PUT /identity/v3/projects/c30e59eecc944c30837fc6164c4a89ea/users/33737a66aafb4884b23f9680a8d4060f/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:50.106119 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c476e5fc-1b80-4327-897b-ce67565d0a5b admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:50.113966 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54717840-d92b-40ba-8124-ad0a6cbb63ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=10e761f41b8949e787f4fb828407283e, user_id=bdc73f789dc944638aaaea07234c5dda, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:50.121642 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54717840-d92b-40ba-8124-ad0a6cbb63ce admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:50.173241 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c476e5fc-1b80-4327-897b-ce67565d0a5b admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:50.178221 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f539dee9-876c-4546-a1e5-63f5fc70c942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:50.178899 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f539dee9-876c-4546-a1e5-63f5fc70c942 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:50.179193 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f539dee9-876c-4546-a1e5-63f5fc70c942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:50.179472 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f539dee9-876c-4546-a1e5-63f5fc70c942 admin admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:50.180167 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f539dee9-876c-4546-a1e5-63f5fc70c942 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:50.182400 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f539dee9-876c-4546-a1e5-63f5fc70c942 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:50.200124 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1063/2098] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:28:49 2022] PUT /identity/v3/projects/10e761f41b8949e787f4fb828407283e/users/bdc73f789dc944638aaaea07234c5dda/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:50.202284 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee54f142-87b5-48cd-9bde-d1df2eaf3bdb None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:50.202284 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee54f142-87b5-48cd-9bde-d1df2eaf3bdb None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:50.202284 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee54f142-87b5-48cd-9bde-d1df2eaf3bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:50.232636 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-ee54f142-87b5-48cd-9bde-d1df2eaf3bdb None None] MFA Rules not processed for user `33737a66aafb4884b23f9680a8d4060f`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:50.233881 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-ee54f142-87b5-48cd-9bde-d1df2eaf3bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:50.289415 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1036/2099] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:28:50 2022] POST /identity/v3/users => generated 305 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:28:50.289415 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f539dee9-876c-4546-a1e5-63f5fc70c942 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:50.298273 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f539dee9-876c-4546-a1e5-63f5fc70c942 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:50.305532 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1064/2100] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:28:50.306461 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee54f142-87b5-48cd-9bde-d1df2eaf3bdb None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:50.359068 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8d5b51-ef47-4f8d-800f-833464d83a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:50.359750 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d5b51-ef47-4f8d-800f-833464d83a36 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:50.360039 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d5b51-ef47-4f8d-800f-833464d83a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:50.360316 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d5b51-ef47-4f8d-800f-833464d83a36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:50.361113 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8d5b51-ef47-4f8d-800f-833464d83a36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:50.362862 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8d5b51-ef47-4f8d-800f-833464d83a36 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:50.373098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1037/2101] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:28:50 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:28:50.374780 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8d5b51-ef47-4f8d-800f-833464d83a36 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:50.375951 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-2f8d5b51-ef47-4f8d-800f-833464d83a36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:50.378003 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee54f142-87b5-48cd-9bde-d1df2eaf3bdb None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:50.382635 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551dcebd-0e7d-44ae-ac9c-8ca56aea5693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:50.383449 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551dcebd-0e7d-44ae-ac9c-8ca56aea5693 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:50.383739 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551dcebd-0e7d-44ae-ac9c-8ca56aea5693 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:50.384699 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551dcebd-0e7d-44ae-ac9c-8ca56aea5693 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:50.385964 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551dcebd-0e7d-44ae-ac9c-8ca56aea5693 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:50.387621 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551dcebd-0e7d-44ae-ac9c-8ca56aea5693 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:50.398037 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1065/2102] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:28:50 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:28:50.399154 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551dcebd-0e7d-44ae-ac9c-8ca56aea5693 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:50.406448 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:50.412182 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551dcebd-0e7d-44ae-ac9c-8ca56aea5693 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:50.418859 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6dfd7f-0e99-40d4-a732-9f37444dfc2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:50.419821 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6dfd7f-0e99-40d4-a732-9f37444dfc2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:50.420162 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6dfd7f-0e99-40d4-a732-9f37444dfc2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:50.420484 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6dfd7f-0e99-40d4-a732-9f37444dfc2a admin admin] PATH_INFO: `/v3/projects/10e761f41b8949e787f4fb828407283e/users/bdc73f789dc944638aaaea07234c5dda/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:50.422592 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6dfd7f-0e99-40d4-a732-9f37444dfc2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=10e761f41b8949e787f4fb828407283e, user_id=bdc73f789dc944638aaaea07234c5dda, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:50.425373 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6dfd7f-0e99-40d4-a732-9f37444dfc2a admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:50.435919 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1066/2103] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:28:50 2022] PUT /identity/v3/projects/10e761f41b8949e787f4fb828407283e/users/bdc73f789dc944638aaaea07234c5dda/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:50.438424 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6dfd7f-0e99-40d4-a732-9f37444dfc2a admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:50.439898 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8d5b51-ef47-4f8d-800f-833464d83a36 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:50.498841 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ba69e6-16c6-4fdf-ac8a-0b32b210b2e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:50.500090 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ba69e6-16c6-4fdf-ac8a-0b32b210b2e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:50.500594 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ba69e6-16c6-4fdf-ac8a-0b32b210b2e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:50.500878 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ba69e6-16c6-4fdf-ac8a-0b32b210b2e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:50.505943 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:50.516754 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6dfd7f-0e99-40d4-a732-9f37444dfc2a admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:50.521606 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bae93d-ad74-4e2a-9b67-09136722dc17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:50.522531 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bae93d-ad74-4e2a-9b67-09136722dc17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:50.522966 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bae93d-ad74-4e2a-9b67-09136722dc17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:50.523248 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bae93d-ad74-4e2a-9b67-09136722dc17 admin admin] PATH_INFO: `/v3/projects/42f20841d5624b93a8f8b34e8f7257c0/users/4b83982c1641438e860a5ccc8d09bccb/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:50.539377 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bae93d-ad74-4e2a-9b67-09136722dc17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42f20841d5624b93a8f8b34e8f7257c0, user_id=4b83982c1641438e860a5ccc8d09bccb, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:50.542289 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bae93d-ad74-4e2a-9b67-09136722dc17 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:50.563259 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ba69e6-16c6-4fdf-ac8a-0b32b210b2e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:50.564745 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ba69e6-16c6-4fdf-ac8a-0b32b210b2e5 service cinder] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:50.575353 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1038/2104] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:28:50 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:28:50.576696 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dee713-40fe-44b3-8870-eb4ff0f10582 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:50.577213 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dee713-40fe-44b3-8870-eb4ff0f10582 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:50.577672 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dee713-40fe-44b3-8870-eb4ff0f10582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:50.615273 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1067/2105] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:28:50 2022] PUT /identity/v3/projects/42f20841d5624b93a8f8b34e8f7257c0/users/4b83982c1641438e860a5ccc8d09bccb/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:50.615445 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-d9dee713-40fe-44b3-8870-eb4ff0f10582 None None] MFA Rules not processed for user `bdc73f789dc944638aaaea07234c5dda`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:50.615445 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-d9dee713-40fe-44b3-8870-eb4ff0f10582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:50.616304 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bae93d-ad74-4e2a-9b67-09136722dc17 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:50.686768 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bae93d-ad74-4e2a-9b67-09136722dc17 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:50.691688 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cdf121-f44e-4f08-8ede-fb52d0632d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:50.692355 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cdf121-f44e-4f08-8ede-fb52d0632d82 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:50.692665 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cdf121-f44e-4f08-8ede-fb52d0632d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:50.692946 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cdf121-f44e-4f08-8ede-fb52d0632d82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:50.693861 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cdf121-f44e-4f08-8ede-fb52d0632d82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:50.695462 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cdf121-f44e-4f08-8ede-fb52d0632d82 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:50.697286 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1039/2106] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:50 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:28:50.712363 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1068/2107] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:28:50 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:28:50.716964 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dee713-40fe-44b3-8870-eb4ff0f10582 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:50.728601 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dee713-40fe-44b3-8870-eb4ff0f10582 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:50.735926 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d297d07-8dd8-4c06-b3ff-6fd3525593e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:50.735926 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d297d07-8dd8-4c06-b3ff-6fd3525593e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:50.735926 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d297d07-8dd8-4c06-b3ff-6fd3525593e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:50.736208 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d297d07-8dd8-4c06-b3ff-6fd3525593e9 admin admin] PATH_INFO: `/v3/projects/42f20841d5624b93a8f8b34e8f7257c0/users/4b83982c1641438e860a5ccc8d09bccb/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:50.738018 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d297d07-8dd8-4c06-b3ff-6fd3525593e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42f20841d5624b93a8f8b34e8f7257c0, user_id=4b83982c1641438e860a5ccc8d09bccb, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:50.740713 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d297d07-8dd8-4c06-b3ff-6fd3525593e9 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:50.754350 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1040/2108] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:28:50 2022] PUT /identity/v3/projects/42f20841d5624b93a8f8b34e8f7257c0/users/4b83982c1641438e860a5ccc8d09bccb/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:50.754554 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cdf121-f44e-4f08-8ede-fb52d0632d82 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:50.756165 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-c4cdf121-f44e-4f08-8ede-fb52d0632d82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:50.765099 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc2719-0efd-49b6-8403-fd7ee6018db5 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:50.765393 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc2719-0efd-49b6-8403-fd7ee6018db5 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:50.765673 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc2719-0efd-49b6-8403-fd7ee6018db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:50.799142 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-18bc2719-0efd-49b6-8403-fd7ee6018db5 None None] MFA Rules not processed for user `4b83982c1641438e860a5ccc8d09bccb`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:50.800425 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-18bc2719-0efd-49b6-8403-fd7ee6018db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:50.822497 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cdf121-f44e-4f08-8ede-fb52d0632d82 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:50.847653 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:50.895572 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1041/2109] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:50 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:28:50.898699 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:50.917705 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be5800b-970f-4ce1-9fc1-67d63bd86f39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:50.918382 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5800b-970f-4ce1-9fc1-67d63bd86f39 service cinder] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:50.918672 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5800b-970f-4ce1-9fc1-67d63bd86f39 service cinder] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:50.918948 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5800b-970f-4ce1-9fc1-67d63bd86f39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:50.979441 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be5800b-970f-4ce1-9fc1-67d63bd86f39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:50.980943 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be5800b-970f-4ce1-9fc1-67d63bd86f39 service cinder] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:50.991076 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1069/2110] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:28:50 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:28:51.204954 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bc2719-0efd-49b6-8403-fd7ee6018db5 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:51.215263 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bc2719-0efd-49b6-8403-fd7ee6018db5 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:51.218920 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5766dc73-9b7f-45d2-ae43-f5efa54374aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:51.219568 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5766dc73-9b7f-45d2-ae43-f5efa54374aa service cinder] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:51.226427 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5766dc73-9b7f-45d2-ae43-f5efa54374aa service cinder] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:51.226427 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5766dc73-9b7f-45d2-ae43-f5efa54374aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:51.231494 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5766dc73-9b7f-45d2-ae43-f5efa54374aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:51.233132 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5766dc73-9b7f-45d2-ae43-f5efa54374aa service cinder] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:51.243892 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1042/2111] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:28:51 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:28:51.251798 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:51.453639 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be5800b-970f-4ce1-9fc1-67d63bd86f39 service cinder] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:51.462272 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be5800b-970f-4ce1-9fc1-67d63bd86f39 service cinder] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:51.466826 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35340ccc-84d8-4828-b2c5-bf2080672ef6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:51.467472 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35340ccc-84d8-4828-b2c5-bf2080672ef6 service cinder] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:51.467757 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35340ccc-84d8-4828-b2c5-bf2080672ef6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:51.468038 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35340ccc-84d8-4828-b2c5-bf2080672ef6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:51.526516 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35340ccc-84d8-4828-b2c5-bf2080672ef6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:51.528050 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35340ccc-84d8-4828-b2c5-bf2080672ef6 service cinder] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:51.537892 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1070/2112] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:28:51 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:28:51.590142 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:51.623275 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5766dc73-9b7f-45d2-ae43-f5efa54374aa service cinder] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:51.632574 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5766dc73-9b7f-45d2-ae43-f5efa54374aa service cinder] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:51.636334 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be70979b-10b3-4d24-82d5-f5d09be696ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:51.637062 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be70979b-10b3-4d24-82d5-f5d09be696ce service cinder] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:51.637352 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be70979b-10b3-4d24-82d5-f5d09be696ce service cinder] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:51.637631 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be70979b-10b3-4d24-82d5-f5d09be696ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:51.695613 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be70979b-10b3-4d24-82d5-f5d09be696ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:51.697191 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be70979b-10b3-4d24-82d5-f5d09be696ce service cinder] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:51.706888 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1043/2113] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:28:51 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:28:51.934294 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35340ccc-84d8-4828-b2c5-bf2080672ef6 service cinder] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:51.935782 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-35340ccc-84d8-4828-b2c5-bf2080672ef6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:51.946482 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:52.013806 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35340ccc-84d8-4828-b2c5-bf2080672ef6 service cinder] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:52.019084 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4059491a-dd55-4bcb-9fd7-86875f78534c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:52.020005 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4059491a-dd55-4bcb-9fd7-86875f78534c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:52.020968 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4059491a-dd55-4bcb-9fd7-86875f78534c admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:52.021257 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4059491a-dd55-4bcb-9fd7-86875f78534c admin admin] PATH_INFO: `/v3/users/da6b5526bcfc4d56a3a3f41eaadc9f7d` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:52.022499 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4059491a-dd55-4bcb-9fd7-86875f78534c admin admin] RBAC: Authorizing `identity:delete_user(user_id=da6b5526bcfc4d56a3a3f41eaadc9f7d)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:52.024491 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4059491a-dd55-4bcb-9fd7-86875f78534c admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:52.210932 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-4059491a-dd55-4bcb-9fd7-86875f78534c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da6b5526bcfc4d56a3a3f41eaadc9f7d'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:52.226155 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-4059491a-dd55-4bcb-9fd7-86875f78534c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da6b5526bcfc4d56a3a3f41eaadc9f7d'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:52.303184 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-4059491a-dd55-4bcb-9fd7-86875f78534c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da6b5526bcfc4d56a3a3f41eaadc9f7d'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:52.306722 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-4059491a-dd55-4bcb-9fd7-86875f78534c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da6b5526bcfc4d56a3a3f41eaadc9f7d'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:52.314325 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-4059491a-dd55-4bcb-9fd7-86875f78534c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da6b5526bcfc4d56a3a3f41eaadc9f7d'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:52.322647 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1071/2114] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:28:51 2022] DELETE /identity/v3/users/da6b5526bcfc4d56a3a3f41eaadc9f7d => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:52.327385 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916df6a-7113-46f2-a6a5-73f462ee1f6c None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:52.327790 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916df6a-7113-46f2-a6a5-73f462ee1f6c None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:52.335587 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916df6a-7113-46f2-a6a5-73f462ee1f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:52.347854 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-7916df6a-7113-46f2-a6a5-73f462ee1f6c None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:52.349205 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-7916df6a-7113-46f2-a6a5-73f462ee1f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:52.415852 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1044/2115] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:28:52.426410 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4059491a-dd55-4bcb-9fd7-86875f78534c admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:52.428410 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-4059491a-dd55-4bcb-9fd7-86875f78534c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:52.505638 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4059491a-dd55-4bcb-9fd7-86875f78534c admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:52.509591 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349f9e98-a90c-48d0-9481-20cdc60d5337 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:52.510274 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f9e98-a90c-48d0-9481-20cdc60d5337 service neutron] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:52.510573 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f9e98-a90c-48d0-9481-20cdc60d5337 service neutron] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:52.510868 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f9e98-a90c-48d0-9481-20cdc60d5337 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:52.523325 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349f9e98-a90c-48d0-9481-20cdc60d5337 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:52.524431 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349f9e98-a90c-48d0-9481-20cdc60d5337 service neutron] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:52.536648 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1072/2116] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:28:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:28:53.014248 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7916df6a-7113-46f2-a6a5-73f462ee1f6c None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:53.016517 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-7916df6a-7113-46f2-a6a5-73f462ee1f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:53.033069 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7916df6a-7113-46f2-a6a5-73f462ee1f6c None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:53.053095 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b62a63-8bc5-42ca-b19d-13254a4683df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:53.054507 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b62a63-8bc5-42ca-b19d-13254a4683df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:53.054855 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b62a63-8bc5-42ca-b19d-13254a4683df admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:53.055070 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b62a63-8bc5-42ca-b19d-13254a4683df admin admin] PATH_INFO: `/v3/projects/778c91d98f0a4fbfa890057b0000f10a` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:53.056654 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b62a63-8bc5-42ca-b19d-13254a4683df admin admin] RBAC: Authorizing `identity:delete_project(project_id=778c91d98f0a4fbfa890057b0000f10a)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:53.058854 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b62a63-8bc5-42ca-b19d-13254a4683df admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:53.203246 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:53.219537 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-79b62a63-8bc5-42ca-b19d-13254a4683df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '778c91d98f0a4fbfa890057b0000f10a', 'request_id': 'req-79b62a63-8bc5-42ca-b19d-13254a4683df'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:53.304589 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-79b62a63-8bc5-42ca-b19d-13254a4683df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '778c91d98f0a4fbfa890057b0000f10a', 'request_id': 'req-79b62a63-8bc5-42ca-b19d-13254a4683df'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:53.313502 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-79b62a63-8bc5-42ca-b19d-13254a4683df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '778c91d98f0a4fbfa890057b0000f10a', 'request_id': 'req-79b62a63-8bc5-42ca-b19d-13254a4683df'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:53.327825 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-79b62a63-8bc5-42ca-b19d-13254a4683df admin admin] The token cache is being invalidate because project 778c91d98f0a4fbfa890057b0000f10a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:28:53.328142 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-79b62a63-8bc5-42ca-b19d-13254a4683df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:53.328787 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-79b62a63-8bc5-42ca-b19d-13254a4683df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:53.330359 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1045/2117] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:28:53 2022] DELETE /identity/v3/projects/778c91d98f0a4fbfa890057b0000f10a => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:53.335331 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026b4477-1b84-428f-8082-6fa430402d36 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:53.335625 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026b4477-1b84-428f-8082-6fa430402d36 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:53.335906 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026b4477-1b84-428f-8082-6fa430402d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:53.355784 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-026b4477-1b84-428f-8082-6fa430402d36 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:53.357060 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-026b4477-1b84-428f-8082-6fa430402d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:53.380281 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:53.422016 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1073/2118] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:28:53.430069 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c562b97f-4663-4d6a-9b99-ba67ef857fc9 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:53.430413 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c562b97f-4663-4d6a-9b99-ba67ef857fc9 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:53.430726 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c562b97f-4663-4d6a-9b99-ba67ef857fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:53.456020 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-c562b97f-4663-4d6a-9b99-ba67ef857fc9 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:53.457416 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-c562b97f-4663-4d6a-9b99-ba67ef857fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:53.469353 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1046/2119] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:28:53.473423 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026b4477-1b84-428f-8082-6fa430402d36 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:53.487889 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026b4477-1b84-428f-8082-6fa430402d36 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:53.492942 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab1f913-4fb8-4cd7-acac-40ab19e93011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:53.494169 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab1f913-4fb8-4cd7-acac-40ab19e93011 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:53.494169 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab1f913-4fb8-4cd7-acac-40ab19e93011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:53.494412 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab1f913-4fb8-4cd7-acac-40ab19e93011 admin admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:53.496036 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab1f913-4fb8-4cd7-acac-40ab19e93011 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:28:53.496616 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab1f913-4fb8-4cd7-acac-40ab19e93011 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:53.498540 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab1f913-4fb8-4cd7-acac-40ab19e93011 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:53.511492 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1074/2120] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:28:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:28:53.516476 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c562b97f-4663-4d6a-9b99-ba67ef857fc9 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:53.527566 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c562b97f-4663-4d6a-9b99-ba67ef857fc9 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:53.533796 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c40204a-18bb-4ec4-b8cd-0a0bd3e0e49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:53.534517 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c40204a-18bb-4ec4-b8cd-0a0bd3e0e49d admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:53.535250 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c40204a-18bb-4ec4-b8cd-0a0bd3e0e49d admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:53.535250 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c40204a-18bb-4ec4-b8cd-0a0bd3e0e49d admin admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:53.536641 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c40204a-18bb-4ec4-b8cd-0a0bd3e0e49d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:53.538628 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c40204a-18bb-4ec4-b8cd-0a0bd3e0e49d admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:53.673298 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1047/2121] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:28:53 2022] POST /identity/v3/projects => generated 369 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:28:53.677200 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab1f913-4fb8-4cd7-acac-40ab19e93011 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:53.689195 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab1f913-4fb8-4cd7-acac-40ab19e93011 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:53.775582 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64dea4c-df07-43f1-b314-06119c49c9b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:53.776523 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64dea4c-df07-43f1-b314-06119c49c9b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:53.777437 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64dea4c-df07-43f1-b314-06119c49c9b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:53.777735 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64dea4c-df07-43f1-b314-06119c49c9b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:53.778453 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64dea4c-df07-43f1-b314-06119c49c9b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:53.780562 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64dea4c-df07-43f1-b314-06119c49c9b6 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:53.889933 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1075/2122] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:28:53 2022] POST /identity/v3/users => generated 285 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:28:53.894191 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c40204a-18bb-4ec4-b8cd-0a0bd3e0e49d admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:53.908894 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c40204a-18bb-4ec4-b8cd-0a0bd3e0e49d admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:53.915722 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d78b68-b6a0-495a-af19-fba5ebd80782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:53.916492 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d78b68-b6a0-495a-af19-fba5ebd80782 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:53.916847 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d78b68-b6a0-495a-af19-fba5ebd80782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:53.917348 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d78b68-b6a0-495a-af19-fba5ebd80782 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:53.918303 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d78b68-b6a0-495a-af19-fba5ebd80782 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:53.920643 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d78b68-b6a0-495a-af19-fba5ebd80782 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:53.934365 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1048/2123] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:28:53 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:28:53.938343 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64dea4c-df07-43f1-b314-06119c49c9b6 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:53.949665 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64dea4c-df07-43f1-b314-06119c49c9b6 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:53.956129 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0fc79a-fc53-44d9-b488-1a78538f3238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:53.956924 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0fc79a-fc53-44d9-b488-1a78538f3238 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:53.957216 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0fc79a-fc53-44d9-b488-1a78538f3238 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:53.957946 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0fc79a-fc53-44d9-b488-1a78538f3238 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:53.958249 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0fc79a-fc53-44d9-b488-1a78538f3238 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:53.960383 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0fc79a-fc53-44d9-b488-1a78538f3238 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:53.971790 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1076/2124] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:28:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:28:53.975859 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d78b68-b6a0-495a-af19-fba5ebd80782 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:54.001471 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d78b68-b6a0-495a-af19-fba5ebd80782 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:54.006857 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c453d596-bddc-4725-8408-c99fffe676da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:54.007769 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c453d596-bddc-4725-8408-c99fffe676da admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:54.008054 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c453d596-bddc-4725-8408-c99fffe676da admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:54.008342 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c453d596-bddc-4725-8408-c99fffe676da admin admin] PATH_INFO: `/v3/projects/ce6fadeffb9442fb83b5f1162d71e91e/users/6630f77b1dba4187a267773246bd2194/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:54.027686 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c453d596-bddc-4725-8408-c99fffe676da admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce6fadeffb9442fb83b5f1162d71e91e, user_id=6630f77b1dba4187a267773246bd2194, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:54.030732 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c453d596-bddc-4725-8408-c99fffe676da admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:54.121101 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1049/2125] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:28:53 2022] PUT /identity/v3/projects/ce6fadeffb9442fb83b5f1162d71e91e/users/6630f77b1dba4187a267773246bd2194/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:54.134857 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385510e6-af9c-45b2-995f-d56394f6f60e None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:54.135162 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385510e6-af9c-45b2-995f-d56394f6f60e None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:54.135960 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385510e6-af9c-45b2-995f-d56394f6f60e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:54.169136 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-385510e6-af9c-45b2-995f-d56394f6f60e None None] MFA Rules not processed for user `6630f77b1dba4187a267773246bd2194`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:54.171339 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-385510e6-af9c-45b2-995f-d56394f6f60e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:54.250683 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1077/2126] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:54 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:28:54.262583 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c453d596-bddc-4725-8408-c99fffe676da admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:54.264453 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-c453d596-bddc-4725-8408-c99fffe676da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:54.336006 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c453d596-bddc-4725-8408-c99fffe676da admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:54.340303 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f63d0b7-4671-49b7-ada9-46c39d2e4593 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9672794bc1ca4dcd8259d51d9290384b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:54.341303 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63d0b7-4671-49b7-ada9-46c39d2e4593 service glance] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:54.341691 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63d0b7-4671-49b7-ada9-46c39d2e4593 service glance] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:54.342098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63d0b7-4671-49b7-ada9-46c39d2e4593 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:54.354826 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f63d0b7-4671-49b7-ada9-46c39d2e4593 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:54.356648 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f63d0b7-4671-49b7-ada9-46c39d2e4593 service glance] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:54.368333 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1050/2127] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:28:54 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:28:55.923800 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385510e6-af9c-45b2-995f-d56394f6f60e None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:55.925399 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-385510e6-af9c-45b2-995f-d56394f6f60e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:55.949861 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:56.001257 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385510e6-af9c-45b2-995f-d56394f6f60e None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:56.006247 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35455d4d-60e9-4694-94e6-8f0f56c67386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:56.007223 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35455d4d-60e9-4694-94e6-8f0f56c67386 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:56.007537 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35455d4d-60e9-4694-94e6-8f0f56c67386 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:56.007813 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35455d4d-60e9-4694-94e6-8f0f56c67386 admin admin] PATH_INFO: `/v3/users/33737a66aafb4884b23f9680a8d4060f` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:56.009069 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35455d4d-60e9-4694-94e6-8f0f56c67386 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33737a66aafb4884b23f9680a8d4060f)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:56.012180 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35455d4d-60e9-4694-94e6-8f0f56c67386 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:56.175138 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-35455d4d-60e9-4694-94e6-8f0f56c67386 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33737a66aafb4884b23f9680a8d4060f'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:56.187382 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-35455d4d-60e9-4694-94e6-8f0f56c67386 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33737a66aafb4884b23f9680a8d4060f'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:56.257218 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-35455d4d-60e9-4694-94e6-8f0f56c67386 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33737a66aafb4884b23f9680a8d4060f'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:56.257877 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-35455d4d-60e9-4694-94e6-8f0f56c67386 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33737a66aafb4884b23f9680a8d4060f'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:56.266336 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-35455d4d-60e9-4694-94e6-8f0f56c67386 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33737a66aafb4884b23f9680a8d4060f'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:56.274376 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1078/2128] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:28:55 2022] DELETE /identity/v3/users/33737a66aafb4884b23f9680a8d4060f => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:56.280260 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b778065-a33e-49aa-92f7-6481f1c59806 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:56.285408 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b778065-a33e-49aa-92f7-6481f1c59806 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:56.285408 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b778065-a33e-49aa-92f7-6481f1c59806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:56.301118 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-0b778065-a33e-49aa-92f7-6481f1c59806 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:56.306561 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-0b778065-a33e-49aa-92f7-6481f1c59806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:56.359836 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:56.373041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1051/2129] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:28:56.383805 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35455d4d-60e9-4694-94e6-8f0f56c67386 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:56.385841 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-35455d4d-60e9-4694-94e6-8f0f56c67386 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:56.452248 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35455d4d-60e9-4694-94e6-8f0f56c67386 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:56.456823 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d9faad-8535-4bdc-9bed-bd822f56754a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:56.457625 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9faad-8535-4bdc-9bed-bd822f56754a service neutron] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:56.457915 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9faad-8535-4bdc-9bed-bd822f56754a service neutron] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:56.458190 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9faad-8535-4bdc-9bed-bd822f56754a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:56.473188 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d9faad-8535-4bdc-9bed-bd822f56754a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:56.476712 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d9faad-8535-4bdc-9bed-bd822f56754a service neutron] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:56.493412 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1079/2130] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:28:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:28:57.020458 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b778065-a33e-49aa-92f7-6481f1c59806 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:57.022471 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-0b778065-a33e-49aa-92f7-6481f1c59806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:57.038576 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b778065-a33e-49aa-92f7-6481f1c59806 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:57.044656 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee31d660-14f2-4efd-b97d-cb0b5d5c05fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:57.045841 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee31d660-14f2-4efd-b97d-cb0b5d5c05fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:57.045976 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee31d660-14f2-4efd-b97d-cb0b5d5c05fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:57.046453 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee31d660-14f2-4efd-b97d-cb0b5d5c05fe admin admin] PATH_INFO: `/v3/projects/c30e59eecc944c30837fc6164c4a89ea` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:57.051250 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee31d660-14f2-4efd-b97d-cb0b5d5c05fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=c30e59eecc944c30837fc6164c4a89ea)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:57.055303 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee31d660-14f2-4efd-b97d-cb0b5d5c05fe admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:57.184750 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-ee31d660-14f2-4efd-b97d-cb0b5d5c05fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c30e59eecc944c30837fc6164c4a89ea', 'request_id': 'req-ee31d660-14f2-4efd-b97d-cb0b5d5c05fe'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:57.250699 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-ee31d660-14f2-4efd-b97d-cb0b5d5c05fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c30e59eecc944c30837fc6164c4a89ea', 'request_id': 'req-ee31d660-14f2-4efd-b97d-cb0b5d5c05fe'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:57.253327 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d9faad-8535-4bdc-9bed-bd822f56754a service neutron] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:57.254929 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-30d9faad-8535-4bdc-9bed-bd822f56754a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:57.259604 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-ee31d660-14f2-4efd-b97d-cb0b5d5c05fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c30e59eecc944c30837fc6164c4a89ea', 'request_id': 'req-ee31d660-14f2-4efd-b97d-cb0b5d5c05fe'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:57.270712 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d9faad-8535-4bdc-9bed-bd822f56754a service neutron] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:57.274070 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-ee31d660-14f2-4efd-b97d-cb0b5d5c05fe admin admin] The token cache is being invalidate because project c30e59eecc944c30837fc6164c4a89ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:28:57.275059 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-ee31d660-14f2-4efd-b97d-cb0b5d5c05fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:57.275289 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-ee31d660-14f2-4efd-b97d-cb0b5d5c05fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:57.278582 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1052/2131] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:28:57 2022] DELETE /identity/v3/projects/c30e59eecc944c30837fc6164c4a89ea => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:57.284044 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1bcd2e-1353-4fa2-a89b-b36c6df6381d None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:57.285076 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1bcd2e-1353-4fa2-a89b-b36c6df6381d None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:57.285076 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1bcd2e-1353-4fa2-a89b-b36c6df6381d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:57.304623 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-fc1bcd2e-1353-4fa2-a89b-b36c6df6381d None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:57.305927 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-fc1bcd2e-1353-4fa2-a89b-b36c6df6381d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:57.319433 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:57.336310 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc4c748-9b69-474e-a042-7a14d14808ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:57.337229 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc4c748-9b69-474e-a042-7a14d14808ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:57.338276 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc4c748-9b69-474e-a042-7a14d14808ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:57.338276 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc4c748-9b69-474e-a042-7a14d14808ac admin admin] PATH_INFO: `/v3/users/1b67f011b29c4985bdbb5c212babc43a` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:57.339775 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc4c748-9b69-474e-a042-7a14d14808ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b67f011b29c4985bdbb5c212babc43a)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:57.341884 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc4c748-9b69-474e-a042-7a14d14808ac admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:57.374915 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1053/2132] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:28:57.383568 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e4ecea-36f2-488b-8271-6971cb74e1e5 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:57.383863 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e4ecea-36f2-488b-8271-6971cb74e1e5 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:57.384154 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e4ecea-36f2-488b-8271-6971cb74e1e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:57.404823 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-08e4ecea-36f2-488b-8271-6971cb74e1e5 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:57.406089 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-08e4ecea-36f2-488b-8271-6971cb74e1e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:57.414429 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1054/2133] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:28:57.420170 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e4ecea-36f2-488b-8271-6971cb74e1e5 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:57.429047 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e4ecea-36f2-488b-8271-6971cb74e1e5 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:57.434595 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8591b2ca-7225-47a9-b7a7-6ea17f270a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:57.435562 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8591b2ca-7225-47a9-b7a7-6ea17f270a96 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:57.435717 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8591b2ca-7225-47a9-b7a7-6ea17f270a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:57.435970 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8591b2ca-7225-47a9-b7a7-6ea17f270a96 admin admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:57.436836 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8591b2ca-7225-47a9-b7a7-6ea17f270a96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:28:57.437193 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8591b2ca-7225-47a9-b7a7-6ea17f270a96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:57.439046 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8591b2ca-7225-47a9-b7a7-6ea17f270a96 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:57.458563 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1055/2134] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:28:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:28:57.459501 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8591b2ca-7225-47a9-b7a7-6ea17f270a96 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:57.473590 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8591b2ca-7225-47a9-b7a7-6ea17f270a96 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:57.480301 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd26712-53d0-499a-abf8-14af232df193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:57.481064 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd26712-53d0-499a-abf8-14af232df193 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:57.481358 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd26712-53d0-499a-abf8-14af232df193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:57.481644 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd26712-53d0-499a-abf8-14af232df193 admin admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:57.482354 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd26712-53d0-499a-abf8-14af232df193 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:57.484337 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd26712-53d0-499a-abf8-14af232df193 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:57.537342 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-bcc4c748-9b69-474e-a042-7a14d14808ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b67f011b29c4985bdbb5c212babc43a'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:57.550653 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-bcc4c748-9b69-474e-a042-7a14d14808ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b67f011b29c4985bdbb5c212babc43a'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:57.608740 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1056/2135] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:28:57 2022] POST /identity/v3/projects => generated 373 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:28:57.612574 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd26712-53d0-499a-abf8-14af232df193 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:57.663173 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:57.664420 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-bcc4c748-9b69-474e-a042-7a14d14808ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b67f011b29c4985bdbb5c212babc43a'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:57.665717 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-bcc4c748-9b69-474e-a042-7a14d14808ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b67f011b29c4985bdbb5c212babc43a'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:57.680504 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-bcc4c748-9b69-474e-a042-7a14d14808ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b67f011b29c4985bdbb5c212babc43a'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:57.692302 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1080/2136] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:28:57 2022] DELETE /identity/v3/users/1b67f011b29c4985bdbb5c212babc43a => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:28:57.697943 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a30d329-d12f-4cb5-bfd6-59d512eecc92 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:57.698815 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a30d329-d12f-4cb5-bfd6-59d512eecc92 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:57.699597 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a30d329-d12f-4cb5-bfd6-59d512eecc92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:57.704904 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd26712-53d0-499a-abf8-14af232df193 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:57.735866 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-1a30d329-d12f-4cb5-bfd6-59d512eecc92 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:57.738045 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-1a30d329-d12f-4cb5-bfd6-59d512eecc92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:57.809254 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4951027-f72d-452e-ba26-6594a0c62e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:57.809882 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4951027-f72d-452e-ba26-6594a0c62e99 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:57.810173 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4951027-f72d-452e-ba26-6594a0c62e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:57.810456 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4951027-f72d-452e-ba26-6594a0c62e99 admin admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:57.811181 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4951027-f72d-452e-ba26-6594a0c62e99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:57.813535 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4951027-f72d-452e-ba26-6594a0c62e99 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:57.861176 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1081/2137] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:28:57.874910 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a30d329-d12f-4cb5-bfd6-59d512eecc92 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:57.876920 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-1a30d329-d12f-4cb5-bfd6-59d512eecc92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:57.916961 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1057/2138] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:28:57 2022] POST /identity/v3/users => generated 287 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:28:57.918895 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4951027-f72d-452e-ba26-6594a0c62e99 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:57.920953 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-d4951027-f72d-452e-ba26-6594a0c62e99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:57.935688 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4951027-f72d-452e-ba26-6594a0c62e99 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:57.942052 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba9d692-7e85-4732-97bf-80cb22eec509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:57.942747 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba9d692-7e85-4732-97bf-80cb22eec509 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:57.943034 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba9d692-7e85-4732-97bf-80cb22eec509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:57.943372 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba9d692-7e85-4732-97bf-80cb22eec509 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:57.944172 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba9d692-7e85-4732-97bf-80cb22eec509 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:57.945940 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba9d692-7e85-4732-97bf-80cb22eec509 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:57.958304 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1058/2139] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:28:57 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:28:57.962566 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba9d692-7e85-4732-97bf-80cb22eec509 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:57.975945 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba9d692-7e85-4732-97bf-80cb22eec509 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:57.977087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a30d329-d12f-4cb5-bfd6-59d512eecc92 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:57.981719 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d054afc3-2294-4767-8acc-84a1f2ab26c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:57.982412 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d054afc3-2294-4767-8acc-84a1f2ab26c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:57.982697 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d054afc3-2294-4767-8acc-84a1f2ab26c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:57.982977 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d054afc3-2294-4767-8acc-84a1f2ab26c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:57.987329 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2d3bc5-766e-4fbf-816f-1f5115056d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:57.988853 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d3bc5-766e-4fbf-816f-1f5115056d06 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:57.989228 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d3bc5-766e-4fbf-816f-1f5115056d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:57.989975 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d3bc5-766e-4fbf-816f-1f5115056d06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:57.992825 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2d3bc5-766e-4fbf-816f-1f5115056d06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:57.993543 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d054afc3-2294-4767-8acc-84a1f2ab26c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:57.995118 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d054afc3-2294-4767-8acc-84a1f2ab26c7 service neutron] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:57.995969 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2d3bc5-766e-4fbf-816f-1f5115056d06 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:58.007106 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1082/2140] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:28:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:28:58.015810 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1059/2141] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:28:57 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:28:58.022178 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d054afc3-2294-4767-8acc-84a1f2ab26c7 service neutron] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:58.041990 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:58.054052 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d054afc3-2294-4767-8acc-84a1f2ab26c7 service neutron] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:58.062481 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0ff5a5-aff7-467d-a2ea-139e9d90a6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:58.064114 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ff5a5-aff7-467d-a2ea-139e9d90a6ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:58.064663 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ff5a5-aff7-467d-a2ea-139e9d90a6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:58.065205 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ff5a5-aff7-467d-a2ea-139e9d90a6ee admin admin] PATH_INFO: `/v3/projects/7c79897f1af044b6a3491af12e0c4a59/users/7e016879344249269eab84058a383e51/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:58.097201 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0ff5a5-aff7-467d-a2ea-139e9d90a6ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c79897f1af044b6a3491af12e0c4a59, user_id=7e016879344249269eab84058a383e51, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:58.102690 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0ff5a5-aff7-467d-a2ea-139e9d90a6ee admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:58.201035 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1083/2142] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:28:58 2022] PUT /identity/v3/projects/7c79897f1af044b6a3491af12e0c4a59/users/7e016879344249269eab84058a383e51/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:58.211691 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6027c9-f57a-4589-8bf4-5ee01bbcadbe None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:58.211989 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6027c9-f57a-4589-8bf4-5ee01bbcadbe None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:58.212269 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6027c9-f57a-4589-8bf4-5ee01bbcadbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:58.241909 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-0a6027c9-f57a-4589-8bf4-5ee01bbcadbe None None] MFA Rules not processed for user `7e016879344249269eab84058a383e51`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:58.243177 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-0a6027c9-f57a-4589-8bf4-5ee01bbcadbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:58.331409 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1060/2143] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:28:58.352028 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0ff5a5-aff7-467d-a2ea-139e9d90a6ee admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:58.354612 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-dc0ff5a5-aff7-467d-a2ea-139e9d90a6ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:58.430345 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0ff5a5-aff7-467d-a2ea-139e9d90a6ee admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:58.433289 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a7a974-2a9e-40ae-85ef-b7d96e1c5f72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:58.433969 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a7a974-2a9e-40ae-85ef-b7d96e1c5f72 service cinder] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:58.434255 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a7a974-2a9e-40ae-85ef-b7d96e1c5f72 service cinder] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:58.434529 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a7a974-2a9e-40ae-85ef-b7d96e1c5f72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:58.444468 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a7a974-2a9e-40ae-85ef-b7d96e1c5f72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:58.445933 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a7a974-2a9e-40ae-85ef-b7d96e1c5f72 service cinder] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:58.458008 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1084/2144] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:28:58 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:28:58.490110 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6027c9-f57a-4589-8bf4-5ee01bbcadbe None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:58.542668 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:58.562615 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6027c9-f57a-4589-8bf4-5ee01bbcadbe None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:58.567600 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a7a974-2a9e-40ae-85ef-b7d96e1c5f72 service cinder] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:58.568161 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b3b8d9-dd7f-45b8-b96f-e4e88e84cdc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:58.569102 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b3b8d9-dd7f-45b8-b96f-e4e88e84cdc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:58.569404 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b3b8d9-dd7f-45b8-b96f-e4e88e84cdc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:58.569657 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b3b8d9-dd7f-45b8-b96f-e4e88e84cdc0 admin admin] PATH_INFO: `/v3/users/7e016879344249269eab84058a383e51` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:58.571145 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b3b8d9-dd7f-45b8-b96f-e4e88e84cdc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e016879344249269eab84058a383e51)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:58.580931 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b3b8d9-dd7f-45b8-b96f-e4e88e84cdc0 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:58.598598 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-13a7a974-2a9e-40ae-85ef-b7d96e1c5f72 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:58.615697 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a7a974-2a9e-40ae-85ef-b7d96e1c5f72 service cinder] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:58.621067 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f31466-94c5-45fa-8209-e31e7502b06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:58.621983 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f31466-94c5-45fa-8209-e31e7502b06a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:58.622263 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f31466-94c5-45fa-8209-e31e7502b06a admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:58.622536 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f31466-94c5-45fa-8209-e31e7502b06a admin admin] PATH_INFO: `/v3/projects/828af27280204701b39fa3ed06fc9bdc` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:58.623775 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f31466-94c5-45fa-8209-e31e7502b06a admin admin] RBAC: Authorizing `identity:delete_project(project_id=828af27280204701b39fa3ed06fc9bdc)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:58.626024 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f31466-94c5-45fa-8209-e31e7502b06a admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:58.729974 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-21b3b8d9-dd7f-45b8-b96f-e4e88e84cdc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e016879344249269eab84058a383e51'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:58.741447 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-94f31466-94c5-45fa-8209-e31e7502b06a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '828af27280204701b39fa3ed06fc9bdc', 'request_id': 'req-94f31466-94c5-45fa-8209-e31e7502b06a'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:58.746611 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-21b3b8d9-dd7f-45b8-b96f-e4e88e84cdc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e016879344249269eab84058a383e51'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:58.747517 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-21b3b8d9-dd7f-45b8-b96f-e4e88e84cdc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e016879344249269eab84058a383e51'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:58.751914 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-94f31466-94c5-45fa-8209-e31e7502b06a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '828af27280204701b39fa3ed06fc9bdc', 'request_id': 'req-94f31466-94c5-45fa-8209-e31e7502b06a'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:58.756796 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-21b3b8d9-dd7f-45b8-b96f-e4e88e84cdc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e016879344249269eab84058a383e51'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:58.757801 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-21b3b8d9-dd7f-45b8-b96f-e4e88e84cdc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e016879344249269eab84058a383e51'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:58.762735 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-94f31466-94c5-45fa-8209-e31e7502b06a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '828af27280204701b39fa3ed06fc9bdc', 'request_id': 'req-94f31466-94c5-45fa-8209-e31e7502b06a'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:58.824938 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1061/2145] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:28:58 2022] DELETE /identity/v3/users/7e016879344249269eab84058a383e51 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:58.830374 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb0d420-9b99-4661-959f-79dd685451ba None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:58.830744 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb0d420-9b99-4661-959f-79dd685451ba None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:58.831128 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb0d420-9b99-4661-959f-79dd685451ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:58.856536 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-8fb0d420-9b99-4661-959f-79dd685451ba None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:28:58.857901 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-8fb0d420-9b99-4661-959f-79dd685451ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:58.910322 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-94f31466-94c5-45fa-8209-e31e7502b06a admin admin] The token cache is being invalidate because project 828af27280204701b39fa3ed06fc9bdc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:28:58.910322 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-94f31466-94c5-45fa-8209-e31e7502b06a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:58.912088 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-94f31466-94c5-45fa-8209-e31e7502b06a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:58.915090 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1085/2146] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:28:58 2022] DELETE /identity/v3/projects/828af27280204701b39fa3ed06fc9bdc => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:28:58.922779 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f31466-94c5-45fa-8209-e31e7502b06a admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:58.924758 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-94f31466-94c5-45fa-8209-e31e7502b06a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:58.932036 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1062/2147] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:28:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:28:58.942845 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb0d420-9b99-4661-959f-79dd685451ba None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:28:58.944272 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-8fb0d420-9b99-4661-959f-79dd685451ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:28:59.012887 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f31466-94c5-45fa-8209-e31e7502b06a admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:59.013817 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb0d420-9b99-4661-959f-79dd685451ba None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:28:59.017807 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f0e2ce-cb04-4679-acbf-ff109a910d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:59.017976 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1321be8-ab6f-432f-a390-53a6d7c1e23d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:28:59.018667 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1321be8-ab6f-432f-a390-53a6d7c1e23d service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:59.018767 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f0e2ce-cb04-4679-acbf-ff109a910d61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:28:59.018987 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f0e2ce-cb04-4679-acbf-ff109a910d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:59.019094 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1321be8-ab6f-432f-a390-53a6d7c1e23d service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:28:59.019274 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f0e2ce-cb04-4679-acbf-ff109a910d61 admin admin] PATH_INFO: `/v3/users/4b83982c1641438e860a5ccc8d09bccb` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:59.019440 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1321be8-ab6f-432f-a390-53a6d7c1e23d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:28:59.020561 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f0e2ce-cb04-4679-acbf-ff109a910d61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b83982c1641438e860a5ccc8d09bccb)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:59.022548 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f0e2ce-cb04-4679-acbf-ff109a910d61 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:59.028887 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:28:59.031265 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1321be8-ab6f-432f-a390-53a6d7c1e23d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:28:59.032795 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1321be8-ab6f-432f-a390-53a6d7c1e23d service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:28:59.045568 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1063/2148] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:28:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:28:59.247348 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-61f0e2ce-cb04-4679-acbf-ff109a910d61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b83982c1641438e860a5ccc8d09bccb'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:59.272417 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-61f0e2ce-cb04-4679-acbf-ff109a910d61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b83982c1641438e860a5ccc8d09bccb'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:59.356199 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-61f0e2ce-cb04-4679-acbf-ff109a910d61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b83982c1641438e860a5ccc8d09bccb'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:59.357684 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-61f0e2ce-cb04-4679-acbf-ff109a910d61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b83982c1641438e860a5ccc8d09bccb'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:59.370297 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-61f0e2ce-cb04-4679-acbf-ff109a910d61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b83982c1641438e860a5ccc8d09bccb'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:28:59.380245 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1086/2149] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:28:58 2022] DELETE /identity/v3/users/4b83982c1641438e860a5ccc8d09bccb => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:00.064744 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1321be8-ab6f-432f-a390-53a6d7c1e23d service neutron] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:00.077086 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:00.083052 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1321be8-ab6f-432f-a390-53a6d7c1e23d service neutron] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:00.111369 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f0e2ce-cb04-4679-acbf-ff109a910d61 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:00.114729 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-61f0e2ce-cb04-4679-acbf-ff109a910d61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:00.227112 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f0e2ce-cb04-4679-acbf-ff109a910d61 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:00.234275 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c14435-e090-4d18-8f38-b73727325d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:00.235000 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c14435-e090-4d18-8f38-b73727325d91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:00.235285 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c14435-e090-4d18-8f38-b73727325d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:00.240036 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c14435-e090-4d18-8f38-b73727325d91 admin admin] PATH_INFO: `/v3/projects/7c79897f1af044b6a3491af12e0c4a59` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:00.240036 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c14435-e090-4d18-8f38-b73727325d91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c79897f1af044b6a3491af12e0c4a59)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:00.240806 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c14435-e090-4d18-8f38-b73727325d91 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:00.305886 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea0da19-7bf7-468c-b1a1-0d2e6f950d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:00.308201 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea0da19-7bf7-468c-b1a1-0d2e6f950d1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:00.308750 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea0da19-7bf7-468c-b1a1-0d2e6f950d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:00.309113 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea0da19-7bf7-468c-b1a1-0d2e6f950d1b admin admin] PATH_INFO: `/v3/projects/42f20841d5624b93a8f8b34e8f7257c0` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:00.310907 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea0da19-7bf7-468c-b1a1-0d2e6f950d1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=42f20841d5624b93a8f8b34e8f7257c0)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:00.312965 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea0da19-7bf7-468c-b1a1-0d2e6f950d1b admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:00.402537 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-16c14435-e090-4d18-8f38-b73727325d91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c79897f1af044b6a3491af12e0c4a59', 'request_id': 'req-16c14435-e090-4d18-8f38-b73727325d91'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:00.412203 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-16c14435-e090-4d18-8f38-b73727325d91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c79897f1af044b6a3491af12e0c4a59', 'request_id': 'req-16c14435-e090-4d18-8f38-b73727325d91'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:00.419389 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-16c14435-e090-4d18-8f38-b73727325d91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c79897f1af044b6a3491af12e0c4a59', 'request_id': 'req-16c14435-e090-4d18-8f38-b73727325d91'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:00.450760 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-5ea0da19-7bf7-468c-b1a1-0d2e6f950d1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42f20841d5624b93a8f8b34e8f7257c0', 'request_id': 'req-5ea0da19-7bf7-468c-b1a1-0d2e6f950d1b'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:00.549165 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-16c14435-e090-4d18-8f38-b73727325d91 admin admin] The token cache is being invalidate because project 7c79897f1af044b6a3491af12e0c4a59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:29:00.549473 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-16c14435-e090-4d18-8f38-b73727325d91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:00.550185 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-16c14435-e090-4d18-8f38-b73727325d91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:00.551843 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1087/2150] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:29:00 2022] DELETE /identity/v3/projects/7c79897f1af044b6a3491af12e0c4a59 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:29:00.601300 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-5ea0da19-7bf7-468c-b1a1-0d2e6f950d1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42f20841d5624b93a8f8b34e8f7257c0', 'request_id': 'req-5ea0da19-7bf7-468c-b1a1-0d2e6f950d1b'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:00.612501 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-5ea0da19-7bf7-468c-b1a1-0d2e6f950d1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42f20841d5624b93a8f8b34e8f7257c0', 'request_id': 'req-5ea0da19-7bf7-468c-b1a1-0d2e6f950d1b'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:00.680964 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-5ea0da19-7bf7-468c-b1a1-0d2e6f950d1b admin admin] The token cache is being invalidate because project 42f20841d5624b93a8f8b34e8f7257c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:29:00.681492 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-5ea0da19-7bf7-468c-b1a1-0d2e6f950d1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:00.683121 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-5ea0da19-7bf7-468c-b1a1-0d2e6f950d1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:00.690327 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1064/2151] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:29:00 2022] DELETE /identity/v3/projects/42f20841d5624b93a8f8b34e8f7257c0 => generated 0 bytes in 623 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:00.692298 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d1339-8965-4196-b1b7-3943775a51bd None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:00.692724 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d1339-8965-4196-b1b7-3943775a51bd None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:00.693117 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d1339-8965-4196-b1b7-3943775a51bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:00.738941 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:00.751311 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-168d1339-8965-4196-b1b7-3943775a51bd None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:00.753740 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-168d1339-8965-4196-b1b7-3943775a51bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:00.890523 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1088/2152] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:29:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:00.903932 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b654982-81cf-4c1f-95cf-7a309dfc0162 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:00.904222 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b654982-81cf-4c1f-95cf-7a309dfc0162 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:00.904544 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b654982-81cf-4c1f-95cf-7a309dfc0162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:00.924109 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-7b654982-81cf-4c1f-95cf-7a309dfc0162 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:00.925470 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-7b654982-81cf-4c1f-95cf-7a309dfc0162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:00.934692 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1065/2153] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:29:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:00.939161 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168d1339-8965-4196-b1b7-3943775a51bd None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:00.950922 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168d1339-8965-4196-b1b7-3943775a51bd None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:00.956526 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a016f95d-6d29-476f-be06-317add94f2d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:00.957434 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a016f95d-6d29-476f-be06-317add94f2d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:00.957720 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a016f95d-6d29-476f-be06-317add94f2d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:00.958110 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a016f95d-6d29-476f-be06-317add94f2d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:00.958871 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a016f95d-6d29-476f-be06-317add94f2d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:29:00.959186 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a016f95d-6d29-476f-be06-317add94f2d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:00.961350 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a016f95d-6d29-476f-be06-317add94f2d6 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:00.978721 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1089/2154] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:29:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:29:00.987647 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b654982-81cf-4c1f-95cf-7a309dfc0162 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:00.998135 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b654982-81cf-4c1f-95cf-7a309dfc0162 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:01.004351 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc690a2e-2d52-4683-a02b-de79530c943d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:01.005063 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc690a2e-2d52-4683-a02b-de79530c943d admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:01.005552 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc690a2e-2d52-4683-a02b-de79530c943d admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:01.005835 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc690a2e-2d52-4683-a02b-de79530c943d admin admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:01.006555 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc690a2e-2d52-4683-a02b-de79530c943d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:01.008777 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc690a2e-2d52-4683-a02b-de79530c943d admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:01.116714 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1066/2155] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:29:00 2022] POST /identity/v3/projects => generated 369 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:29:01.122763 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a016f95d-6d29-476f-be06-317add94f2d6 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:01.133772 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a016f95d-6d29-476f-be06-317add94f2d6 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:01.215567 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8012bea6-8f56-44bf-b8f3-1de0dfc619b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:01.216300 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012bea6-8f56-44bf-b8f3-1de0dfc619b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:01.238703 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012bea6-8f56-44bf-b8f3-1de0dfc619b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:01.238703 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012bea6-8f56-44bf-b8f3-1de0dfc619b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:01.238703 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8012bea6-8f56-44bf-b8f3-1de0dfc619b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:01.238703 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8012bea6-8f56-44bf-b8f3-1de0dfc619b4 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:01.327346 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1090/2156] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:29:01 2022] POST /identity/v3/users => generated 285 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:29:01.329480 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc690a2e-2d52-4683-a02b-de79530c943d admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:01.359330 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc690a2e-2d52-4683-a02b-de79530c943d admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:01.383242 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fbb5cb-76d9-4d2e-bfa6-9d3e64454d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:01.383995 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fbb5cb-76d9-4d2e-bfa6-9d3e64454d1c admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:01.388020 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fbb5cb-76d9-4d2e-bfa6-9d3e64454d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:01.388020 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fbb5cb-76d9-4d2e-bfa6-9d3e64454d1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:01.388020 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fbb5cb-76d9-4d2e-bfa6-9d3e64454d1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:01.388020 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fbb5cb-76d9-4d2e-bfa6-9d3e64454d1c admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:01.404228 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1067/2157] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:01 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:29:01.409295 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8012bea6-8f56-44bf-b8f3-1de0dfc619b4 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:01.433772 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8012bea6-8f56-44bf-b8f3-1de0dfc619b4 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:01.445941 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2bfcc9-0446-4bd6-b17f-6f0af09efa6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:01.445941 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2bfcc9-0446-4bd6-b17f-6f0af09efa6d admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:01.445941 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2bfcc9-0446-4bd6-b17f-6f0af09efa6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:01.445941 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2bfcc9-0446-4bd6-b17f-6f0af09efa6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:01.446909 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2bfcc9-0446-4bd6-b17f-6f0af09efa6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:01.449986 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2bfcc9-0446-4bd6-b17f-6f0af09efa6d admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:01.476657 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1091/2158] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:01 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:29:01.486063 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fbb5cb-76d9-4d2e-bfa6-9d3e64454d1c admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:01.498190 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fbb5cb-76d9-4d2e-bfa6-9d3e64454d1c admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:01.508886 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d88942b-29eb-4ff7-accf-913f2ecc20cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:01.508886 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d88942b-29eb-4ff7-accf-913f2ecc20cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:01.509460 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d88942b-29eb-4ff7-accf-913f2ecc20cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:01.509847 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d88942b-29eb-4ff7-accf-913f2ecc20cb admin admin] PATH_INFO: `/v3/projects/02f7adbad51e4f29aae3da01ec334872/users/7dc90a9a937e4414b373982c96027a67/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:01.547951 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d88942b-29eb-4ff7-accf-913f2ecc20cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=02f7adbad51e4f29aae3da01ec334872, user_id=7dc90a9a937e4414b373982c96027a67, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:01.554641 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d88942b-29eb-4ff7-accf-913f2ecc20cb admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:01.610859 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1068/2159] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:29:01 2022] PUT /identity/v3/projects/02f7adbad51e4f29aae3da01ec334872/users/7dc90a9a937e4414b373982c96027a67/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:01.616318 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e954f6-066a-46ee-9915-1287c0d93013 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:01.616862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e954f6-066a-46ee-9915-1287c0d93013 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:01.617148 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e954f6-066a-46ee-9915-1287c0d93013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:01.642091 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-f8e954f6-066a-46ee-9915-1287c0d93013 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:01.643701 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-f8e954f6-066a-46ee-9915-1287c0d93013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:01.779368 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:01.848452 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1092/2160] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:29:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:29:01.881319 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d88942b-29eb-4ff7-accf-913f2ecc20cb admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:01.884417 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-5d88942b-29eb-4ff7-accf-913f2ecc20cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:01.921693 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:01.996650 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d88942b-29eb-4ff7-accf-913f2ecc20cb admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:02.002550 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17de0124-1245-4685-bf7a-5540e79df345 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:02.003647 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17de0124-1245-4685-bf7a-5540e79df345 service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:02.004309 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17de0124-1245-4685-bf7a-5540e79df345 service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:02.004763 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17de0124-1245-4685-bf7a-5540e79df345 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:02.023385 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17de0124-1245-4685-bf7a-5540e79df345 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:02.026702 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17de0124-1245-4685-bf7a-5540e79df345 service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:02.046734 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1069/2161] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:29:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:29:02.959066 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2ec7d-6a58-4637-9824-8fd26fdf516c None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:02.959603 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2ec7d-6a58-4637-9824-8fd26fdf516c None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:02.959699 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2ec7d-6a58-4637-9824-8fd26fdf516c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:02.978581 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-7bb2ec7d-6a58-4637-9824-8fd26fdf516c None None] MFA Rules not processed for user `b6bbd45ee6964c4986f3c778ab8ad231`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:02.979908 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-7bb2ec7d-6a58-4637-9824-8fd26fdf516c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:03.049907 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1093/2162] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:29:02 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:03.785099 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cca38f-d464-433c-9f36-c13e0214c2ef None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:03.786069 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cca38f-d464-433c-9f36-c13e0214c2ef None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:03.798357 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cca38f-d464-433c-9f36-c13e0214c2ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:03.857971 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-a1cca38f-d464-433c-9f36-c13e0214c2ef None None] MFA Rules not processed for user `7dc90a9a937e4414b373982c96027a67`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:03.860127 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-a1cca38f-d464-433c-9f36-c13e0214c2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:04.044632 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1070/2163] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:29:03 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:04.045978 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb2ec7d-6a58-4637-9824-8fd26fdf516c None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:04.062586 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb2ec7d-6a58-4637-9824-8fd26fdf516c None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:04.069056 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6209ea4-453a-48f5-a836-0e52fcd89664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:04.069749 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6209ea4-453a-48f5-a836-0e52fcd89664 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:04.070033 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6209ea4-453a-48f5-a836-0e52fcd89664 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:04.070345 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6209ea4-453a-48f5-a836-0e52fcd89664 admin admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:04.071048 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6209ea4-453a-48f5-a836-0e52fcd89664 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:04.074830 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6209ea4-453a-48f5-a836-0e52fcd89664 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:04.204967 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1094/2164] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:29:04 2022] POST /identity/v3/projects => generated 371 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:29:04.209140 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cca38f-d464-433c-9f36-c13e0214c2ef None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:04.220986 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cca38f-d464-433c-9f36-c13e0214c2ef None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:04.323867 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee836a4-250c-4e25-919d-3b9757dd4633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:04.324974 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee836a4-250c-4e25-919d-3b9757dd4633 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:04.325527 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee836a4-250c-4e25-919d-3b9757dd4633 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:04.326083 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee836a4-250c-4e25-919d-3b9757dd4633 admin admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:04.327223 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee836a4-250c-4e25-919d-3b9757dd4633 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:04.329881 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee836a4-250c-4e25-919d-3b9757dd4633 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:04.464170 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1071/2165] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:29:04 2022] POST /identity/v3/users => generated 292 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:29:04.469755 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6209ea4-453a-48f5-a836-0e52fcd89664 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:04.481410 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6209ea4-453a-48f5-a836-0e52fcd89664 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:04.486274 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dad7490-8afc-4f0f-95e5-aa18715e7d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:04.486948 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad7490-8afc-4f0f-95e5-aa18715e7d79 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:04.487261 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad7490-8afc-4f0f-95e5-aa18715e7d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:04.487537 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad7490-8afc-4f0f-95e5-aa18715e7d79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:04.488279 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dad7490-8afc-4f0f-95e5-aa18715e7d79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:04.490000 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dad7490-8afc-4f0f-95e5-aa18715e7d79 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:04.503595 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1095/2166] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:29:04.508314 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee836a4-250c-4e25-919d-3b9757dd4633 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:04.519784 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee836a4-250c-4e25-919d-3b9757dd4633 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:04.524845 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5a7155-c9c0-4580-a91a-8c2597e9da0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:04.525936 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5a7155-c9c0-4580-a91a-8c2597e9da0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:04.526277 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5a7155-c9c0-4580-a91a-8c2597e9da0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:04.526557 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5a7155-c9c0-4580-a91a-8c2597e9da0e admin admin] PATH_INFO: `/v3/projects/50a09afb7faf46f19bea95e1f6cb080e/users/767a9b4eb6ba4237b12b1e2dc0702af1/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:04.539967 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:04.543815 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5a7155-c9c0-4580-a91a-8c2597e9da0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=50a09afb7faf46f19bea95e1f6cb080e, user_id=767a9b4eb6ba4237b12b1e2dc0702af1, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:04.546599 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5a7155-c9c0-4580-a91a-8c2597e9da0e admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:04.604974 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dad7490-8afc-4f0f-95e5-aa18715e7d79 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:04.607213 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-0dad7490-8afc-4f0f-95e5-aa18715e7d79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:04.623619 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dad7490-8afc-4f0f-95e5-aa18715e7d79 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:04.634304 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cd575c-7a1a-4239-8273-b790123e2995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:04.635806 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd575c-7a1a-4239-8273-b790123e2995 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:04.636270 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd575c-7a1a-4239-8273-b790123e2995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:04.636876 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd575c-7a1a-4239-8273-b790123e2995 admin admin] PATH_INFO: `/v3/users/0259af286ce24d928a13b2aeb0700246` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:04.639089 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cd575c-7a1a-4239-8273-b790123e2995 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0259af286ce24d928a13b2aeb0700246)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:04.641366 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cd575c-7a1a-4239-8273-b790123e2995 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:04.652322 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1072/2167] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:29:04 2022] PUT /identity/v3/projects/50a09afb7faf46f19bea95e1f6cb080e/users/767a9b4eb6ba4237b12b1e2dc0702af1/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:04.658386 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5a7155-c9c0-4580-a91a-8c2597e9da0e admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:04.764792 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5a7155-c9c0-4580-a91a-8c2597e9da0e admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:04.769684 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b521a96-b300-4139-9e4a-03dc9269275d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:04.770410 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b521a96-b300-4139-9e4a-03dc9269275d admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:04.770691 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b521a96-b300-4139-9e4a-03dc9269275d admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:04.770968 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b521a96-b300-4139-9e4a-03dc9269275d admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:04.771738 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b521a96-b300-4139-9e4a-03dc9269275d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:04.773442 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b521a96-b300-4139-9e4a-03dc9269275d admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:04.786148 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1073/2168] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:04 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:29:04.792761 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b521a96-b300-4139-9e4a-03dc9269275d admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:04.803448 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b521a96-b300-4139-9e4a-03dc9269275d admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:04.809953 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fb3ea0-2ff9-4cb8-9dd7-9206e8f36985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:04.815895 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fb3ea0-2ff9-4cb8-9dd7-9206e8f36985 admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:04.815895 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fb3ea0-2ff9-4cb8-9dd7-9206e8f36985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:04.815895 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fb3ea0-2ff9-4cb8-9dd7-9206e8f36985 admin admin] PATH_INFO: `/v3/projects/50a09afb7faf46f19bea95e1f6cb080e/users/767a9b4eb6ba4237b12b1e2dc0702af1/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:04.815895 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fb3ea0-2ff9-4cb8-9dd7-9206e8f36985 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50a09afb7faf46f19bea95e1f6cb080e, user_id=767a9b4eb6ba4237b12b1e2dc0702af1, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:04.816383 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fb3ea0-2ff9-4cb8-9dd7-9206e8f36985 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:04.830291 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1074/2169] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:29:04 2022] PUT /identity/v3/projects/50a09afb7faf46f19bea95e1f6cb080e/users/767a9b4eb6ba4237b12b1e2dc0702af1/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:29:04.934179 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-b7cd575c-7a1a-4239-8273-b790123e2995 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0259af286ce24d928a13b2aeb0700246'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:04.953263 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-b7cd575c-7a1a-4239-8273-b790123e2995 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0259af286ce24d928a13b2aeb0700246'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:05.054065 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-b7cd575c-7a1a-4239-8273-b790123e2995 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0259af286ce24d928a13b2aeb0700246'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:05.055255 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-b7cd575c-7a1a-4239-8273-b790123e2995 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0259af286ce24d928a13b2aeb0700246'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:05.063742 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-b7cd575c-7a1a-4239-8273-b790123e2995 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0259af286ce24d928a13b2aeb0700246'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:05.070840 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1096/2170] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:29:04 2022] DELETE /identity/v3/users/0259af286ce24d928a13b2aeb0700246 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:29:05.075901 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1119a8bc-c946-4caf-b73c-2123f0a17dbf None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:05.076193 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1119a8bc-c946-4caf-b73c-2123f0a17dbf None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:05.076503 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1119a8bc-c946-4caf-b73c-2123f0a17dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:05.098076 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-1119a8bc-c946-4caf-b73c-2123f0a17dbf None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:05.099315 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-1119a8bc-c946-4caf-b73c-2123f0a17dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:05.203167 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1075/2171] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:29:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:05.214134 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cd575c-7a1a-4239-8273-b790123e2995 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:05.217862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-b7cd575c-7a1a-4239-8273-b790123e2995 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:05.272135 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:05.315968 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cd575c-7a1a-4239-8273-b790123e2995 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:05.323948 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03298cb4-7010-4654-b0a8-3acabf1386ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:05.326338 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03298cb4-7010-4654-b0a8-3acabf1386ba service neutron] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:05.327350 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03298cb4-7010-4654-b0a8-3acabf1386ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:05.327762 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03298cb4-7010-4654-b0a8-3acabf1386ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:05.352773 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03298cb4-7010-4654-b0a8-3acabf1386ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:05.355682 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03298cb4-7010-4654-b0a8-3acabf1386ba service neutron] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:05.375226 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1097/2172] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:29:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:29:06.024846 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1119a8bc-c946-4caf-b73c-2123f0a17dbf None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:06.026935 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-1119a8bc-c946-4caf-b73c-2123f0a17dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:06.068143 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1119a8bc-c946-4caf-b73c-2123f0a17dbf None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:06.074605 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be02da3-bcc3-4790-a5a3-e930ca399696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:06.075676 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be02da3-bcc3-4790-a5a3-e930ca399696 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:06.076057 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be02da3-bcc3-4790-a5a3-e930ca399696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:06.076510 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be02da3-bcc3-4790-a5a3-e930ca399696 admin admin] PATH_INFO: `/v3/projects/23d613c758b94f498ff7d0d996b7b7db` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:06.078099 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be02da3-bcc3-4790-a5a3-e930ca399696 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23d613c758b94f498ff7d0d996b7b7db)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:06.080195 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be02da3-bcc3-4790-a5a3-e930ca399696 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:06.219935 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-4be02da3-bcc3-4790-a5a3-e930ca399696 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23d613c758b94f498ff7d0d996b7b7db', 'request_id': 'req-4be02da3-bcc3-4790-a5a3-e930ca399696'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:06.231376 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:06.337977 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-4be02da3-bcc3-4790-a5a3-e930ca399696 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23d613c758b94f498ff7d0d996b7b7db', 'request_id': 'req-4be02da3-bcc3-4790-a5a3-e930ca399696'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:06.348813 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-4be02da3-bcc3-4790-a5a3-e930ca399696 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23d613c758b94f498ff7d0d996b7b7db', 'request_id': 'req-4be02da3-bcc3-4790-a5a3-e930ca399696'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:06.371037 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-4be02da3-bcc3-4790-a5a3-e930ca399696 admin admin] The token cache is being invalidate because project 23d613c758b94f498ff7d0d996b7b7db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:29:06.371624 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-4be02da3-bcc3-4790-a5a3-e930ca399696 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:06.372821 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-4be02da3-bcc3-4790-a5a3-e930ca399696 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:06.379321 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1076/2173] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:29:06 2022] DELETE /identity/v3/projects/23d613c758b94f498ff7d0d996b7b7db => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:06.381333 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03298cb4-7010-4654-b0a8-3acabf1386ba service neutron] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:06.413032 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03298cb4-7010-4654-b0a8-3acabf1386ba service neutron] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:06.524897 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78f5d8b-57b3-4e8d-8522-ca076b371c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:06.525876 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78f5d8b-57b3-4e8d-8522-ca076b371c3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:06.526149 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78f5d8b-57b3-4e8d-8522-ca076b371c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:06.526422 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78f5d8b-57b3-4e8d-8522-ca076b371c3f admin admin] PATH_INFO: `/v3/users/bdc73f789dc944638aaaea07234c5dda` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:06.528446 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78f5d8b-57b3-4e8d-8522-ca076b371c3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdc73f789dc944638aaaea07234c5dda)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:06.534541 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78f5d8b-57b3-4e8d-8522-ca076b371c3f admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:06.771284 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633e41f0-0a02-422b-83ed-8b88a4de09d5 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:06.774347 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633e41f0-0a02-422b-83ed-8b88a4de09d5 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:06.774842 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633e41f0-0a02-422b-83ed-8b88a4de09d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:06.811046 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-c78f5d8b-57b3-4e8d-8522-ca076b371c3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdc73f789dc944638aaaea07234c5dda'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:06.843629 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-c78f5d8b-57b3-4e8d-8522-ca076b371c3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdc73f789dc944638aaaea07234c5dda'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:06.856778 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-633e41f0-0a02-422b-83ed-8b88a4de09d5 None None] MFA Rules not processed for user `767a9b4eb6ba4237b12b1e2dc0702af1`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:06.858591 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-633e41f0-0a02-422b-83ed-8b88a4de09d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:06.944269 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-c78f5d8b-57b3-4e8d-8522-ca076b371c3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdc73f789dc944638aaaea07234c5dda'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:06.954175 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-c78f5d8b-57b3-4e8d-8522-ca076b371c3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdc73f789dc944638aaaea07234c5dda'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:06.973860 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-c78f5d8b-57b3-4e8d-8522-ca076b371c3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdc73f789dc944638aaaea07234c5dda'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:07.001972 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1098/2174] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:29:06 2022] DELETE /identity/v3/users/bdc73f789dc944638aaaea07234c5dda => generated 0 bytes in 616 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:07.023472 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1077/2175] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:29:06 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:07.036061 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78f5d8b-57b3-4e8d-8522-ca076b371c3f admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:07.037810 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-c78f5d8b-57b3-4e8d-8522-ca076b371c3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:07.105162 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:07.115584 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78f5d8b-57b3-4e8d-8522-ca076b371c3f admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:07.120587 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006d19e6-d42d-4518-8603-948494f223f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:07.121276 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d19e6-d42d-4518-8603-948494f223f5 service cinder] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:07.121567 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d19e6-d42d-4518-8603-948494f223f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:07.121850 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d19e6-d42d-4518-8603-948494f223f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:07.124854 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-006d19e6-d42d-4518-8603-948494f223f5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:07.216029 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006d19e6-d42d-4518-8603-948494f223f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:07.217639 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006d19e6-d42d-4518-8603-948494f223f5 service cinder] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:07.227812 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1099/2176] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:29:07 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:29:07.723613 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633e41f0-0a02-422b-83ed-8b88a4de09d5 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:07.729624 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-633e41f0-0a02-422b-83ed-8b88a4de09d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:07.752182 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633e41f0-0a02-422b-83ed-8b88a4de09d5 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:07.937601 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df2aa78-5802-487c-8356-7de74b81b4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:07.939701 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df2aa78-5802-487c-8356-7de74b81b4ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:07.940536 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df2aa78-5802-487c-8356-7de74b81b4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:07.941295 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df2aa78-5802-487c-8356-7de74b81b4ba admin admin] PATH_INFO: `/v3/projects/10e761f41b8949e787f4fb828407283e` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:07.945050 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df2aa78-5802-487c-8356-7de74b81b4ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=10e761f41b8949e787f4fb828407283e)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:07.950318 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df2aa78-5802-487c-8356-7de74b81b4ba admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:08.170461 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-2df2aa78-5802-487c-8356-7de74b81b4ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10e761f41b8949e787f4fb828407283e', 'request_id': 'req-2df2aa78-5802-487c-8356-7de74b81b4ba'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:08.263802 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-2df2aa78-5802-487c-8356-7de74b81b4ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10e761f41b8949e787f4fb828407283e', 'request_id': 'req-2df2aa78-5802-487c-8356-7de74b81b4ba'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:08.274351 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-2df2aa78-5802-487c-8356-7de74b81b4ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10e761f41b8949e787f4fb828407283e', 'request_id': 'req-2df2aa78-5802-487c-8356-7de74b81b4ba'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:08.278159 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:08.289727 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-2df2aa78-5802-487c-8356-7de74b81b4ba admin admin] The token cache is being invalidate because project 10e761f41b8949e787f4fb828407283e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:29:08.290118 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-2df2aa78-5802-487c-8356-7de74b81b4ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:08.291153 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-2df2aa78-5802-487c-8356-7de74b81b4ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:08.297718 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1078/2177] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:29:07 2022] DELETE /identity/v3/projects/10e761f41b8949e787f4fb828407283e => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:08.299122 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a31605c-8b80-4699-b39d-4b8d5635d768 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:08.299929 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a31605c-8b80-4699-b39d-4b8d5635d768 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:08.300088 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a31605c-8b80-4699-b39d-4b8d5635d768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:08.360014 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-2a31605c-8b80-4699-b39d-4b8d5635d768 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:08.362325 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-2a31605c-8b80-4699-b39d-4b8d5635d768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:08.468654 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1100/2178] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:29:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:29:08.471539 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586037d0-3354-40c4-9a75-d2760acc068f None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:08.471837 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586037d0-3354-40c4-9a75-d2760acc068f None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:08.472260 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586037d0-3354-40c4-9a75-d2760acc068f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:08.499008 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-586037d0-3354-40c4-9a75-d2760acc068f None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:08.501486 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-586037d0-3354-40c4-9a75-d2760acc068f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:08.517486 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1079/2179] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:29:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:08.521747 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a31605c-8b80-4699-b39d-4b8d5635d768 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:08.539310 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a31605c-8b80-4699-b39d-4b8d5635d768 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:08.549167 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98a2173-2dab-4b5e-9ab5-b523b8e6a5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:08.549998 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98a2173-2dab-4b5e-9ab5-b523b8e6a5a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:08.550320 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98a2173-2dab-4b5e-9ab5-b523b8e6a5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:08.550758 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98a2173-2dab-4b5e-9ab5-b523b8e6a5a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:08.551620 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98a2173-2dab-4b5e-9ab5-b523b8e6a5a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:29:08.552077 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98a2173-2dab-4b5e-9ab5-b523b8e6a5a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:08.554009 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98a2173-2dab-4b5e-9ab5-b523b8e6a5a4 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:08.572955 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1101/2180] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:29:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:29:08.574523 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586037d0-3354-40c4-9a75-d2760acc068f None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:08.585399 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586037d0-3354-40c4-9a75-d2760acc068f None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:08.591844 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a31185-e65c-4767-9832-4ba23faeac7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:08.592587 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a31185-e65c-4767-9832-4ba23faeac7a admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:08.604942 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a31185-e65c-4767-9832-4ba23faeac7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:08.604942 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a31185-e65c-4767-9832-4ba23faeac7a admin admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:08.604942 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a31185-e65c-4767-9832-4ba23faeac7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:08.604942 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a31185-e65c-4767-9832-4ba23faeac7a admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:08.789185 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1080/2181] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:29:08 2022] POST /identity/v3/projects => generated 373 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:29:08.795276 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98a2173-2dab-4b5e-9ab5-b523b8e6a5a4 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:08.812798 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98a2173-2dab-4b5e-9ab5-b523b8e6a5a4 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:08.918801 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca78a60-bde0-413e-868a-326f729b77b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:08.919567 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca78a60-bde0-413e-868a-326f729b77b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:08.919882 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca78a60-bde0-413e-868a-326f729b77b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:08.920136 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca78a60-bde0-413e-868a-326f729b77b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:08.920908 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca78a60-bde0-413e-868a-326f729b77b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:08.923249 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca78a60-bde0-413e-868a-326f729b77b3 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:08.935185 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:09.047698 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1102/2182] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:29:08 2022] POST /identity/v3/users => generated 287 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:29:09.051958 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a31185-e65c-4767-9832-4ba23faeac7a admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:09.064456 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a31185-e65c-4767-9832-4ba23faeac7a admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:09.069845 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f218003f-801a-426a-ab41-f0d8d55eb5ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:09.070539 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f218003f-801a-426a-ab41-f0d8d55eb5ab admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:09.070821 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f218003f-801a-426a-ab41-f0d8d55eb5ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:09.071096 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f218003f-801a-426a-ab41-f0d8d55eb5ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:09.071860 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f218003f-801a-426a-ab41-f0d8d55eb5ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:09.074266 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f218003f-801a-426a-ab41-f0d8d55eb5ab admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:09.090200 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1081/2183] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:09 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:29:09.094877 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca78a60-bde0-413e-868a-326f729b77b3 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:09.107529 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca78a60-bde0-413e-868a-326f729b77b3 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:09.113977 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b215129-0be9-4f76-8266-79f0ffbc064b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:09.115033 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b215129-0be9-4f76-8266-79f0ffbc064b admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:09.115495 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b215129-0be9-4f76-8266-79f0ffbc064b admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:09.115908 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b215129-0be9-4f76-8266-79f0ffbc064b admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:09.117259 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b215129-0be9-4f76-8266-79f0ffbc064b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:09.122463 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b215129-0be9-4f76-8266-79f0ffbc064b admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:09.136474 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1103/2184] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:09 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:29:09.143107 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f218003f-801a-426a-ab41-f0d8d55eb5ab admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:09.161859 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f218003f-801a-426a-ab41-f0d8d55eb5ab admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:09.184317 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f049965-7478-4a17-812b-57f95d20f413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:09.185506 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f049965-7478-4a17-812b-57f95d20f413 admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:09.188485 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f049965-7478-4a17-812b-57f95d20f413 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:09.188485 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f049965-7478-4a17-812b-57f95d20f413 admin admin] PATH_INFO: `/v3/projects/68b6d8ce5d9f4fb5b5864806e71a0b7d/users/b8ec583292f943b48f5229823c84fb9e/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:09.235056 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f049965-7478-4a17-812b-57f95d20f413 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68b6d8ce5d9f4fb5b5864806e71a0b7d, user_id=b8ec583292f943b48f5229823c84fb9e, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:09.241086 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f049965-7478-4a17-812b-57f95d20f413 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:09.308871 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b215129-0be9-4f76-8266-79f0ffbc064b admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:09.314050 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-6b215129-0be9-4f76-8266-79f0ffbc064b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:09.395764 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b215129-0be9-4f76-8266-79f0ffbc064b admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:09.412306 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1082/2185] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:29:09 2022] PUT /identity/v3/projects/68b6d8ce5d9f4fb5b5864806e71a0b7d/users/b8ec583292f943b48f5229823c84fb9e/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:09.419912 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3244cad-1098-42bb-bbb1-07b82f98f829 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:09.420516 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3244cad-1098-42bb-bbb1-07b82f98f829 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:09.421004 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3244cad-1098-42bb-bbb1-07b82f98f829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:09.454982 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-e3244cad-1098-42bb-bbb1-07b82f98f829 None None] MFA Rules not processed for user `b8ec583292f943b48f5229823c84fb9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:09.456269 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-e3244cad-1098-42bb-bbb1-07b82f98f829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:09.482814 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf89a3f-fcfc-4227-8c62-3a0ceda619ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:09.483487 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf89a3f-fcfc-4227-8c62-3a0ceda619ed service cinder] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:09.483765 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf89a3f-fcfc-4227-8c62-3a0ceda619ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:09.484043 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf89a3f-fcfc-4227-8c62-3a0ceda619ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:09.486978 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-9cf89a3f-fcfc-4227-8c62-3a0ceda619ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:09.555963 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1083/2186] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:29:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:09.563455 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3244cad-1098-42bb-bbb1-07b82f98f829 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:09.604683 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf89a3f-fcfc-4227-8c62-3a0ceda619ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:09.609737 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf89a3f-fcfc-4227-8c62-3a0ceda619ed service cinder] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:09.628974 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1104/2187] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:29:09 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:29:09.689801 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3244cad-1098-42bb-bbb1-07b82f98f829 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:09.697980 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a032dc9d-edfc-4704-9cc9-b54e1f61fd39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:09.698795 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a032dc9d-edfc-4704-9cc9-b54e1f61fd39 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:09.699184 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a032dc9d-edfc-4704-9cc9-b54e1f61fd39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:09.699525 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a032dc9d-edfc-4704-9cc9-b54e1f61fd39 admin admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:09.700370 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a032dc9d-edfc-4704-9cc9-b54e1f61fd39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:09.704598 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a032dc9d-edfc-4704-9cc9-b54e1f61fd39 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:09.943785 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1084/2188] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:29:09 2022] POST /identity/v3/projects => generated 371 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:29:09.949427 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf89a3f-fcfc-4227-8c62-3a0ceda619ed service cinder] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:09.972680 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf89a3f-fcfc-4227-8c62-3a0ceda619ed service cinder] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:10.042430 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:10.055071 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32eb7f0-ab7e-4717-80e9-290e96f54fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:10.056005 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32eb7f0-ab7e-4717-80e9-290e96f54fb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:10.056361 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32eb7f0-ab7e-4717-80e9-290e96f54fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:10.056777 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32eb7f0-ab7e-4717-80e9-290e96f54fb3 admin admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:10.058247 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32eb7f0-ab7e-4717-80e9-290e96f54fb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:10.060282 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32eb7f0-ab7e-4717-80e9-290e96f54fb3 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:10.157791 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1105/2189] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:29:09 2022] POST /identity/v3/users => generated 292 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:29:10.162123 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a032dc9d-edfc-4704-9cc9-b54e1f61fd39 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:10.173910 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a032dc9d-edfc-4704-9cc9-b54e1f61fd39 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:10.179791 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ad3c9f-cc79-4049-9418-48e06be15c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:10.180620 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad3c9f-cc79-4049-9418-48e06be15c86 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:10.180924 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad3c9f-cc79-4049-9418-48e06be15c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:10.181315 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad3c9f-cc79-4049-9418-48e06be15c86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:10.182185 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ad3c9f-cc79-4049-9418-48e06be15c86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:10.184465 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ad3c9f-cc79-4049-9418-48e06be15c86 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:10.186300 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:10.197753 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1085/2190] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:29:10.202339 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32eb7f0-ab7e-4717-80e9-290e96f54fb3 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:10.214395 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32eb7f0-ab7e-4717-80e9-290e96f54fb3 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:10.219311 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90d11da-e5d2-4d7e-baf4-35bcc15af0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:10.220276 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d11da-e5d2-4d7e-baf4-35bcc15af0b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:10.220780 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d11da-e5d2-4d7e-baf4-35bcc15af0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:10.220954 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d11da-e5d2-4d7e-baf4-35bcc15af0b4 admin admin] PATH_INFO: `/v3/projects/8c8c5a86ab5447c2b588c66b2dac79f1/users/899bd990d3474858b5fe3c8db218ee70/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:10.239129 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90d11da-e5d2-4d7e-baf4-35bcc15af0b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c8c5a86ab5447c2b588c66b2dac79f1, user_id=899bd990d3474858b5fe3c8db218ee70, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:10.241969 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90d11da-e5d2-4d7e-baf4-35bcc15af0b4 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:10.333390 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1106/2191] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:29:10 2022] PUT /identity/v3/projects/8c8c5a86ab5447c2b588c66b2dac79f1/users/899bd990d3474858b5fe3c8db218ee70/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:10.337894 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ad3c9f-cc79-4049-9418-48e06be15c86 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:10.349370 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ad3c9f-cc79-4049-9418-48e06be15c86 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:10.412513 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1154e3-c4d2-4ae6-a83e-3541093a38d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:10.413204 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1154e3-c4d2-4ae6-a83e-3541093a38d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:10.413487 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1154e3-c4d2-4ae6-a83e-3541093a38d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:10.413763 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1154e3-c4d2-4ae6-a83e-3541093a38d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:10.414497 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1154e3-c4d2-4ae6-a83e-3541093a38d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:10.417050 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1154e3-c4d2-4ae6-a83e-3541093a38d7 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:10.429494 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1086/2192] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:10 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:29:10.434620 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90d11da-e5d2-4d7e-baf4-35bcc15af0b4 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:10.445469 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90d11da-e5d2-4d7e-baf4-35bcc15af0b4 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:10.450821 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc175166-40fd-4ad2-b2a1-5ce40d53a658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:10.451723 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc175166-40fd-4ad2-b2a1-5ce40d53a658 admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:10.452009 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc175166-40fd-4ad2-b2a1-5ce40d53a658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:10.452283 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc175166-40fd-4ad2-b2a1-5ce40d53a658 admin admin] PATH_INFO: `/v3/projects/8c8c5a86ab5447c2b588c66b2dac79f1/users/899bd990d3474858b5fe3c8db218ee70/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:10.454399 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc175166-40fd-4ad2-b2a1-5ce40d53a658 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c8c5a86ab5447c2b588c66b2dac79f1, user_id=899bd990d3474858b5fe3c8db218ee70, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:10.457363 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc175166-40fd-4ad2-b2a1-5ce40d53a658 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:10.470098 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1107/2193] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:29:10 2022] PUT /identity/v3/projects/8c8c5a86ab5447c2b588c66b2dac79f1/users/899bd990d3474858b5fe3c8db218ee70/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:10.481803 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c82b7f6-2cfc-4c78-a16e-822a4f0e6fb3 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:10.482089 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c82b7f6-2cfc-4c78-a16e-822a4f0e6fb3 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:10.482368 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c82b7f6-2cfc-4c78-a16e-822a4f0e6fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:10.513314 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-4c82b7f6-2cfc-4c78-a16e-822a4f0e6fb3 None None] MFA Rules not processed for user `899bd990d3474858b5fe3c8db218ee70`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:10.514590 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-4c82b7f6-2cfc-4c78-a16e-822a4f0e6fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:10.596474 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1087/2194] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:29:10 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:29:10.611721 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc175166-40fd-4ad2-b2a1-5ce40d53a658 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:10.614324 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-fc175166-40fd-4ad2-b2a1-5ce40d53a658 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:10.719087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc175166-40fd-4ad2-b2a1-5ce40d53a658 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:10.725723 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7a7141-36d0-4842-abd0-e6d78701906d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:10.726708 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7a7141-36d0-4842-abd0-e6d78701906d service cinder] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:10.727211 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7a7141-36d0-4842-abd0-e6d78701906d service cinder] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:10.727744 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7a7141-36d0-4842-abd0-e6d78701906d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:10.749351 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7a7141-36d0-4842-abd0-e6d78701906d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:10.752302 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7a7141-36d0-4842-abd0-e6d78701906d service cinder] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:10.772589 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1108/2195] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:29:10 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:29:12.034967 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c82b7f6-2cfc-4c78-a16e-822a4f0e6fb3 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:12.119070 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c82b7f6-2cfc-4c78-a16e-822a4f0e6fb3 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:12.124785 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20656f4-4a4b-4898-b090-aac0a4b188f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:12.125687 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20656f4-4a4b-4898-b090-aac0a4b188f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:12.125973 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20656f4-4a4b-4898-b090-aac0a4b188f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:12.126260 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20656f4-4a4b-4898-b090-aac0a4b188f6 admin admin] PATH_INFO: `/v3/users/b8ec583292f943b48f5229823c84fb9e` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:12.127533 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20656f4-4a4b-4898-b090-aac0a4b188f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8ec583292f943b48f5229823c84fb9e)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:12.129565 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20656f4-4a4b-4898-b090-aac0a4b188f6 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:12.157824 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:12.364196 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-d20656f4-4a4b-4898-b090-aac0a4b188f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8ec583292f943b48f5229823c84fb9e'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:12.378115 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-d20656f4-4a4b-4898-b090-aac0a4b188f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8ec583292f943b48f5229823c84fb9e'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:12.378885 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-d20656f4-4a4b-4898-b090-aac0a4b188f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8ec583292f943b48f5229823c84fb9e'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:12.387282 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-d20656f4-4a4b-4898-b090-aac0a4b188f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8ec583292f943b48f5229823c84fb9e'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:12.388037 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-d20656f4-4a4b-4898-b090-aac0a4b188f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8ec583292f943b48f5229823c84fb9e'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:12.455246 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1088/2196] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:29:12 2022] DELETE /identity/v3/users/b8ec583292f943b48f5229823c84fb9e => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:29:12.460437 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c1d0f8-4d21-4d66-914a-6d7371aebabc None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:12.461408 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c1d0f8-4d21-4d66-914a-6d7371aebabc None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:12.461408 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c1d0f8-4d21-4d66-914a-6d7371aebabc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:12.483576 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-d3c1d0f8-4d21-4d66-914a-6d7371aebabc None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:12.485287 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-d3c1d0f8-4d21-4d66-914a-6d7371aebabc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:12.557621 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1109/2197] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:29:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:12.568563 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20656f4-4a4b-4898-b090-aac0a4b188f6 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:12.570135 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-d20656f4-4a4b-4898-b090-aac0a4b188f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:12.672693 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20656f4-4a4b-4898-b090-aac0a4b188f6 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:12.676567 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4f632c-29a3-4d2f-9f03-945464e58607 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:12.677247 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f632c-29a3-4d2f-9f03-945464e58607 service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:12.677536 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f632c-29a3-4d2f-9f03-945464e58607 service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:12.677816 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f632c-29a3-4d2f-9f03-945464e58607 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:12.688595 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4f632c-29a3-4d2f-9f03-945464e58607 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:12.690076 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4f632c-29a3-4d2f-9f03-945464e58607 service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:12.701235 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1089/2198] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:29:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:29:13.220473 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c1d0f8-4d21-4d66-914a-6d7371aebabc None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:13.222480 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-d3c1d0f8-4d21-4d66-914a-6d7371aebabc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:13.237720 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c1d0f8-4d21-4d66-914a-6d7371aebabc None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:13.245467 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f25cad-7b9b-4d33-a550-bec4c8930360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:13.246761 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f25cad-7b9b-4d33-a550-bec4c8930360 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:13.247045 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f25cad-7b9b-4d33-a550-bec4c8930360 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:13.247325 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f25cad-7b9b-4d33-a550-bec4c8930360 admin admin] PATH_INFO: `/v3/projects/68b6d8ce5d9f4fb5b5864806e71a0b7d` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:13.249028 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f25cad-7b9b-4d33-a550-bec4c8930360 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68b6d8ce5d9f4fb5b5864806e71a0b7d)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:13.251516 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f25cad-7b9b-4d33-a550-bec4c8930360 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:13.445664 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:13.446954 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-e0f25cad-7b9b-4d33-a550-bec4c8930360 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68b6d8ce5d9f4fb5b5864806e71a0b7d', 'request_id': 'req-e0f25cad-7b9b-4d33-a550-bec4c8930360'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:13.461851 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-e0f25cad-7b9b-4d33-a550-bec4c8930360 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68b6d8ce5d9f4fb5b5864806e71a0b7d', 'request_id': 'req-e0f25cad-7b9b-4d33-a550-bec4c8930360'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:13.475084 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-e0f25cad-7b9b-4d33-a550-bec4c8930360 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68b6d8ce5d9f4fb5b5864806e71a0b7d', 'request_id': 'req-e0f25cad-7b9b-4d33-a550-bec4c8930360'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:13.552324 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-e0f25cad-7b9b-4d33-a550-bec4c8930360 admin admin] The token cache is being invalidate because project 68b6d8ce5d9f4fb5b5864806e71a0b7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:29:13.553005 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-e0f25cad-7b9b-4d33-a550-bec4c8930360 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:13.554282 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-e0f25cad-7b9b-4d33-a550-bec4c8930360 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:13.557157 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1110/2199] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:29:13 2022] DELETE /identity/v3/projects/68b6d8ce5d9f4fb5b5864806e71a0b7d => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:29:13.563372 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4f632c-29a3-4d2f-9f03-945464e58607 service neutron] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:13.582981 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4f632c-29a3-4d2f-9f03-945464e58607 service neutron] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:13.662499 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb34c6b-fa64-483d-a746-18fa9af4e197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:13.663450 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb34c6b-fa64-483d-a746-18fa9af4e197 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:13.663741 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb34c6b-fa64-483d-a746-18fa9af4e197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:13.664036 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb34c6b-fa64-483d-a746-18fa9af4e197 admin admin] PATH_INFO: `/v3/users/899bd990d3474858b5fe3c8db218ee70` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:13.665566 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb34c6b-fa64-483d-a746-18fa9af4e197 admin admin] RBAC: Authorizing `identity:delete_user(user_id=899bd990d3474858b5fe3c8db218ee70)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:13.668163 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb34c6b-fa64-483d-a746-18fa9af4e197 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:13.846796 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:13.874093 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-afb34c6b-fa64-483d-a746-18fa9af4e197 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '899bd990d3474858b5fe3c8db218ee70'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:13.897688 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-afb34c6b-fa64-483d-a746-18fa9af4e197 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '899bd990d3474858b5fe3c8db218ee70'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:13.898892 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-afb34c6b-fa64-483d-a746-18fa9af4e197 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '899bd990d3474858b5fe3c8db218ee70'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:13.912784 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-afb34c6b-fa64-483d-a746-18fa9af4e197 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '899bd990d3474858b5fe3c8db218ee70'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:13.914970 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-afb34c6b-fa64-483d-a746-18fa9af4e197 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '899bd990d3474858b5fe3c8db218ee70'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:14.049176 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1090/2200] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:29:13 2022] DELETE /identity/v3/users/899bd990d3474858b5fe3c8db218ee70 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:14.864979 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f25cad-7b9b-4d33-a550-bec4c8930360 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:14.867472 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-e0f25cad-7b9b-4d33-a550-bec4c8930360 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:14.882273 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f25cad-7b9b-4d33-a550-bec4c8930360 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:14.976657 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b23915-7f0a-4fdd-a595-e7e1d2179845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:14.977780 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b23915-7f0a-4fdd-a595-e7e1d2179845 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:14.978069 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b23915-7f0a-4fdd-a595-e7e1d2179845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:14.978370 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b23915-7f0a-4fdd-a595-e7e1d2179845 admin admin] PATH_INFO: `/v3/projects/8c8c5a86ab5447c2b588c66b2dac79f1` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:14.979834 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b23915-7f0a-4fdd-a595-e7e1d2179845 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c8c5a86ab5447c2b588c66b2dac79f1)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:14.982181 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b23915-7f0a-4fdd-a595-e7e1d2179845 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:15.206282 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-d6b23915-7f0a-4fdd-a595-e7e1d2179845 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c8c5a86ab5447c2b588c66b2dac79f1', 'request_id': 'req-d6b23915-7f0a-4fdd-a595-e7e1d2179845'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:15.214654 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-d6b23915-7f0a-4fdd-a595-e7e1d2179845 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c8c5a86ab5447c2b588c66b2dac79f1', 'request_id': 'req-d6b23915-7f0a-4fdd-a595-e7e1d2179845'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:15.221810 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-d6b23915-7f0a-4fdd-a595-e7e1d2179845 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c8c5a86ab5447c2b588c66b2dac79f1', 'request_id': 'req-d6b23915-7f0a-4fdd-a595-e7e1d2179845'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:15.356001 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-d6b23915-7f0a-4fdd-a595-e7e1d2179845 admin admin] The token cache is being invalidate because project 8c8c5a86ab5447c2b588c66b2dac79f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:29:15.356764 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-d6b23915-7f0a-4fdd-a595-e7e1d2179845 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:15.357938 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-d6b23915-7f0a-4fdd-a595-e7e1d2179845 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:15.360884 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1111/2201] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:29:14 2022] DELETE /identity/v3/projects/8c8c5a86ab5447c2b588c66b2dac79f1 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:29:15.368121 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab3b22-69e9-42ac-ba9c-4826f622e3d4 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:15.368929 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab3b22-69e9-42ac-ba9c-4826f622e3d4 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:15.368929 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab3b22-69e9-42ac-ba9c-4826f622e3d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:15.392207 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-4dab3b22-69e9-42ac-ba9c-4826f622e3d4 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:15.393648 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-4dab3b22-69e9-42ac-ba9c-4826f622e3d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:15.506014 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1091/2202] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:29:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:29:15.519219 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2646ab-3c5f-47cc-8252-fc89132b0452 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:15.525045 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2646ab-3c5f-47cc-8252-fc89132b0452 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:15.525490 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2646ab-3c5f-47cc-8252-fc89132b0452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:15.560908 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-db2646ab-3c5f-47cc-8252-fc89132b0452 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:15.562160 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-db2646ab-3c5f-47cc-8252-fc89132b0452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:15.573238 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1112/2203] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:29:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:29:15.577825 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dab3b22-69e9-42ac-ba9c-4826f622e3d4 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:15.590699 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dab3b22-69e9-42ac-ba9c-4826f622e3d4 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:15.596514 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff0a81-f15a-4ad8-ad97-5decbe954143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:15.597249 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff0a81-f15a-4ad8-ad97-5decbe954143 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:15.597535 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff0a81-f15a-4ad8-ad97-5decbe954143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:15.597813 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff0a81-f15a-4ad8-ad97-5decbe954143 admin admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:15.598583 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fff0a81-f15a-4ad8-ad97-5decbe954143 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:29:15.598892 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fff0a81-f15a-4ad8-ad97-5decbe954143 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:15.600559 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fff0a81-f15a-4ad8-ad97-5decbe954143 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:15.623041 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2646ab-3c5f-47cc-8252-fc89132b0452 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:15.629750 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1092/2204] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:29:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:29:15.633648 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2646ab-3c5f-47cc-8252-fc89132b0452 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:15.639581 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6b79c0-23ef-4f2d-a95e-ad8d74c2f8ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:15.641031 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b79c0-23ef-4f2d-a95e-ad8d74c2f8ce admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:15.641031 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b79c0-23ef-4f2d-a95e-ad8d74c2f8ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:15.641031 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b79c0-23ef-4f2d-a95e-ad8d74c2f8ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:15.641578 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6b79c0-23ef-4f2d-a95e-ad8d74c2f8ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:15.643518 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6b79c0-23ef-4f2d-a95e-ad8d74c2f8ce admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:15.777251 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1113/2205] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:29:15 2022] POST /identity/v3/projects => generated 367 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:29:15.779029 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff0a81-f15a-4ad8-ad97-5decbe954143 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:15.799119 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff0a81-f15a-4ad8-ad97-5decbe954143 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:15.878819 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a40379-3980-4071-8d84-f4b06ce5bc62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:15.880122 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a40379-3980-4071-8d84-f4b06ce5bc62 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:15.880511 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a40379-3980-4071-8d84-f4b06ce5bc62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:15.880968 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a40379-3980-4071-8d84-f4b06ce5bc62 admin admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:15.881765 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a40379-3980-4071-8d84-f4b06ce5bc62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:15.883807 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a40379-3980-4071-8d84-f4b06ce5bc62 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:16.012799 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1093/2206] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:29:15 2022] POST /identity/v3/users => generated 284 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:29:16.018430 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6b79c0-23ef-4f2d-a95e-ad8d74c2f8ce admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:16.043969 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6b79c0-23ef-4f2d-a95e-ad8d74c2f8ce admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:16.054781 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9599d7dd-f625-4549-9eb9-c3b9ac4834d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:16.057089 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599d7dd-f625-4549-9eb9-c3b9ac4834d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:16.062521 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599d7dd-f625-4549-9eb9-c3b9ac4834d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:16.062521 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599d7dd-f625-4549-9eb9-c3b9ac4834d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:16.062521 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9599d7dd-f625-4549-9eb9-c3b9ac4834d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:16.062521 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9599d7dd-f625-4549-9eb9-c3b9ac4834d4 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:16.082906 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1114/2207] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:16 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:29:16.091394 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a40379-3980-4071-8d84-f4b06ce5bc62 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:16.107464 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a40379-3980-4071-8d84-f4b06ce5bc62 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:16.113060 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33645966-aa2d-4dfa-85c1-21484773ed1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:16.113750 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33645966-aa2d-4dfa-85c1-21484773ed1a admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:16.114033 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33645966-aa2d-4dfa-85c1-21484773ed1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:16.114458 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33645966-aa2d-4dfa-85c1-21484773ed1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:16.115264 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33645966-aa2d-4dfa-85c1-21484773ed1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:16.117049 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33645966-aa2d-4dfa-85c1-21484773ed1a admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:16.136984 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1094/2208] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:16 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:29:16.141220 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9599d7dd-f625-4549-9eb9-c3b9ac4834d4 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:16.158827 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9599d7dd-f625-4549-9eb9-c3b9ac4834d4 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:16.171532 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6c5b37-1f4e-49b0-9d0e-9d378b313298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:16.172749 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c5b37-1f4e-49b0-9d0e-9d378b313298 admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:16.173098 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c5b37-1f4e-49b0-9d0e-9d378b313298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:16.173441 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c5b37-1f4e-49b0-9d0e-9d378b313298 admin admin] PATH_INFO: `/v3/projects/7df52edbfc594a60b8c175ef4a63875c/users/281e1156a8db4796ac02a2e53173dc9a/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:16.193122 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6c5b37-1f4e-49b0-9d0e-9d378b313298 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7df52edbfc594a60b8c175ef4a63875c, user_id=281e1156a8db4796ac02a2e53173dc9a, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:16.195797 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6c5b37-1f4e-49b0-9d0e-9d378b313298 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:16.199952 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:16.327960 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1115/2209] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:29:16 2022] PUT /identity/v3/projects/7df52edbfc594a60b8c175ef4a63875c/users/281e1156a8db4796ac02a2e53173dc9a/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:29:16.343636 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a0510c-04c8-41b3-8617-7ce296c86785 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:16.344189 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a0510c-04c8-41b3-8617-7ce296c86785 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:16.344700 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a0510c-04c8-41b3-8617-7ce296c86785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:16.379057 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-26a0510c-04c8-41b3-8617-7ce296c86785 None None] MFA Rules not processed for user `281e1156a8db4796ac02a2e53173dc9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:16.380507 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-26a0510c-04c8-41b3-8617-7ce296c86785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:16.385295 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:16.461118 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1095/2210] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:29:16 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:16.466215 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6c5b37-1f4e-49b0-9d0e-9d378b313298 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:16.561974 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6c5b37-1f4e-49b0-9d0e-9d378b313298 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:16.569473 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1e633-4aaf-4570-85e1-8f84184956a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:16.570234 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1e633-4aaf-4570-85e1-8f84184956a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:16.570514 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1e633-4aaf-4570-85e1-8f84184956a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:16.570787 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1e633-4aaf-4570-85e1-8f84184956a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:16.571496 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b1e633-4aaf-4570-85e1-8f84184956a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:16.574011 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b1e633-4aaf-4570-85e1-8f84184956a9 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:16.724797 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1116/2211] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:29:16 2022] POST /identity/v3/projects => generated 367 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:29:16.730335 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a0510c-04c8-41b3-8617-7ce296c86785 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:16.745281 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a0510c-04c8-41b3-8617-7ce296c86785 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:16.822474 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009708b3-abb6-4c5e-a35b-2d0fbdb81d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:16.823671 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009708b3-abb6-4c5e-a35b-2d0fbdb81d11 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:16.824412 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009708b3-abb6-4c5e-a35b-2d0fbdb81d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:16.824947 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009708b3-abb6-4c5e-a35b-2d0fbdb81d11 admin admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:16.826297 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009708b3-abb6-4c5e-a35b-2d0fbdb81d11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:16.830253 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009708b3-abb6-4c5e-a35b-2d0fbdb81d11 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:16.986883 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1096/2212] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:29:16 2022] POST /identity/v3/users => generated 290 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:29:16.991943 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1e633-4aaf-4570-85e1-8f84184956a9 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:17.013484 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1e633-4aaf-4570-85e1-8f84184956a9 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:17.021803 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803f4b82-79bb-41a5-8f42-c7b41d6f4708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:17.025002 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f4b82-79bb-41a5-8f42-c7b41d6f4708 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:17.025617 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f4b82-79bb-41a5-8f42-c7b41d6f4708 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:17.026123 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f4b82-79bb-41a5-8f42-c7b41d6f4708 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:17.027893 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803f4b82-79bb-41a5-8f42-c7b41d6f4708 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:17.031074 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803f4b82-79bb-41a5-8f42-c7b41d6f4708 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:17.052076 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1117/2213] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:16 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:29:17.060231 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009708b3-abb6-4c5e-a35b-2d0fbdb81d11 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:17.081582 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009708b3-abb6-4c5e-a35b-2d0fbdb81d11 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:17.091659 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05210446-2010-47c6-a58f-e1e8dd5b46f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:17.093072 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05210446-2010-47c6-a58f-e1e8dd5b46f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:17.093573 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05210446-2010-47c6-a58f-e1e8dd5b46f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:17.094030 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05210446-2010-47c6-a58f-e1e8dd5b46f6 admin admin] PATH_INFO: `/v3/projects/8d71f4e9434b4612a5afe4763ca28268/users/72defa94e12a43af9ea199cdc064d8c6/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:17.123575 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05210446-2010-47c6-a58f-e1e8dd5b46f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d71f4e9434b4612a5afe4763ca28268, user_id=72defa94e12a43af9ea199cdc064d8c6, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:17.128497 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05210446-2010-47c6-a58f-e1e8dd5b46f6 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:17.263081 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1097/2214] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:29:17 2022] PUT /identity/v3/projects/8d71f4e9434b4612a5afe4763ca28268/users/72defa94e12a43af9ea199cdc064d8c6/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:17.268754 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803f4b82-79bb-41a5-8f42-c7b41d6f4708 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:17.280215 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803f4b82-79bb-41a5-8f42-c7b41d6f4708 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:17.360201 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6c31d9-95de-47aa-9a62-74a4ed4d9748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:17.361258 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c31d9-95de-47aa-9a62-74a4ed4d9748 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:17.361258 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c31d9-95de-47aa-9a62-74a4ed4d9748 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:17.361527 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c31d9-95de-47aa-9a62-74a4ed4d9748 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:17.362273 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6c31d9-95de-47aa-9a62-74a4ed4d9748 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:17.364365 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6c31d9-95de-47aa-9a62-74a4ed4d9748 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:17.381385 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1118/2215] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:17 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:29:17.384027 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05210446-2010-47c6-a58f-e1e8dd5b46f6 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:17.397657 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05210446-2010-47c6-a58f-e1e8dd5b46f6 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:17.403874 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc45b3b-fa15-4ee7-9447-1c5d4274d352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:17.404811 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc45b3b-fa15-4ee7-9447-1c5d4274d352 admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:17.405090 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc45b3b-fa15-4ee7-9447-1c5d4274d352 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:17.405365 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc45b3b-fa15-4ee7-9447-1c5d4274d352 admin admin] PATH_INFO: `/v3/projects/8d71f4e9434b4612a5afe4763ca28268/users/72defa94e12a43af9ea199cdc064d8c6/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:17.407528 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc45b3b-fa15-4ee7-9447-1c5d4274d352 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d71f4e9434b4612a5afe4763ca28268, user_id=72defa94e12a43af9ea199cdc064d8c6, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:17.410520 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc45b3b-fa15-4ee7-9447-1c5d4274d352 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:17.423728 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1098/2216] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:29:17 2022] PUT /identity/v3/projects/8d71f4e9434b4612a5afe4763ca28268/users/72defa94e12a43af9ea199cdc064d8c6/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:17.434773 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19c8f3f-2bc1-4d16-954d-0ef87ed60dcf None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:17.435059 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19c8f3f-2bc1-4d16-954d-0ef87ed60dcf None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:17.435394 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19c8f3f-2bc1-4d16-954d-0ef87ed60dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:17.471752 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-a19c8f3f-2bc1-4d16-954d-0ef87ed60dcf None None] MFA Rules not processed for user `72defa94e12a43af9ea199cdc064d8c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:17.473074 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-a19c8f3f-2bc1-4d16-954d-0ef87ed60dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:17.542958 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:17.564367 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1119/2217] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:29:17 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:17.578336 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc45b3b-fa15-4ee7-9447-1c5d4274d352 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:17.580283 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-7cc45b3b-fa15-4ee7-9447-1c5d4274d352 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:17.683171 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc45b3b-fa15-4ee7-9447-1c5d4274d352 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:17.690332 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ffa2d3-2633-4d08-87d1-e6a2fc71414d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:17.691377 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ffa2d3-2633-4d08-87d1-e6a2fc71414d service cinder] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:17.691828 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ffa2d3-2633-4d08-87d1-e6a2fc71414d service cinder] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:17.692300 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ffa2d3-2633-4d08-87d1-e6a2fc71414d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:17.709394 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ffa2d3-2633-4d08-87d1-e6a2fc71414d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:17.711966 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ffa2d3-2633-4d08-87d1-e6a2fc71414d service cinder] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:17.724478 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:17.727084 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1099/2218] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:29:17 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:29:24.482695 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19c8f3f-2bc1-4d16-954d-0ef87ed60dcf None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:24.580974 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19c8f3f-2bc1-4d16-954d-0ef87ed60dcf None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:24.588553 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97a13c2-e5fe-4534-b223-2b110559ae0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:24.590077 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97a13c2-e5fe-4534-b223-2b110559ae0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:24.590774 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97a13c2-e5fe-4534-b223-2b110559ae0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:24.591568 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97a13c2-e5fe-4534-b223-2b110559ae0a admin admin] PATH_INFO: `/v3/users/281e1156a8db4796ac02a2e53173dc9a` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:24.594051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97a13c2-e5fe-4534-b223-2b110559ae0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=281e1156a8db4796ac02a2e53173dc9a)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:24.596777 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97a13c2-e5fe-4534-b223-2b110559ae0a admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:25.056461 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-c97a13c2-e5fe-4534-b223-2b110559ae0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '281e1156a8db4796ac02a2e53173dc9a'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:25.100043 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-c97a13c2-e5fe-4534-b223-2b110559ae0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '281e1156a8db4796ac02a2e53173dc9a'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:25.206395 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-c97a13c2-e5fe-4534-b223-2b110559ae0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '281e1156a8db4796ac02a2e53173dc9a'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:25.207920 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-c97a13c2-e5fe-4534-b223-2b110559ae0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '281e1156a8db4796ac02a2e53173dc9a'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:25.219794 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-c97a13c2-e5fe-4534-b223-2b110559ae0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '281e1156a8db4796ac02a2e53173dc9a'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:25.226890 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1120/2219] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:29:24 2022] DELETE /identity/v3/users/281e1156a8db4796ac02a2e53173dc9a => generated 0 bytes in 745 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:25.232102 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ff975-7c71-49cb-8ec8-5a8dc5809734 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:25.232828 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ff975-7c71-49cb-8ec8-5a8dc5809734 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:25.232828 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ff975-7c71-49cb-8ec8-5a8dc5809734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:25.252872 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-c56ff975-7c71-49cb-8ec8-5a8dc5809734 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:25.254571 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-c56ff975-7c71-49cb-8ec8-5a8dc5809734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:25.324625 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1100/2220] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:29:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:25.337158 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97a13c2-e5fe-4534-b223-2b110559ae0a admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:25.341015 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-c97a13c2-e5fe-4534-b223-2b110559ae0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:25.414386 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97a13c2-e5fe-4534-b223-2b110559ae0a admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:25.418881 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9516c6e1-5168-4d5c-9264-1ae8a2788a83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:25.419559 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9516c6e1-5168-4d5c-9264-1ae8a2788a83 service neutron] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:25.419847 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9516c6e1-5168-4d5c-9264-1ae8a2788a83 service neutron] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:25.420205 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9516c6e1-5168-4d5c-9264-1ae8a2788a83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:25.431144 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9516c6e1-5168-4d5c-9264-1ae8a2788a83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:25.432853 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9516c6e1-5168-4d5c-9264-1ae8a2788a83 service neutron] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:25.443770 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1121/2221] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:29:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:29:26.575423 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56ff975-7c71-49cb-8ec8-5a8dc5809734 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:26.577284 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-c56ff975-7c71-49cb-8ec8-5a8dc5809734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:26.595567 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56ff975-7c71-49cb-8ec8-5a8dc5809734 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:26.622926 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6673178a-7f73-4740-9e99-f92ecba777d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:26.632977 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6673178a-7f73-4740-9e99-f92ecba777d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:26.633294 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6673178a-7f73-4740-9e99-f92ecba777d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:26.633581 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6673178a-7f73-4740-9e99-f92ecba777d4 admin admin] PATH_INFO: `/v3/projects/7df52edbfc594a60b8c175ef4a63875c` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:26.636015 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6673178a-7f73-4740-9e99-f92ecba777d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7df52edbfc594a60b8c175ef4a63875c)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:26.638736 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6673178a-7f73-4740-9e99-f92ecba777d4 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:26.859717 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-6673178a-7f73-4740-9e99-f92ecba777d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7df52edbfc594a60b8c175ef4a63875c', 'request_id': 'req-6673178a-7f73-4740-9e99-f92ecba777d4'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:27.032849 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-6673178a-7f73-4740-9e99-f92ecba777d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7df52edbfc594a60b8c175ef4a63875c', 'request_id': 'req-6673178a-7f73-4740-9e99-f92ecba777d4'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:27.065012 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-6673178a-7f73-4740-9e99-f92ecba777d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7df52edbfc594a60b8c175ef4a63875c', 'request_id': 'req-6673178a-7f73-4740-9e99-f92ecba777d4'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:27.091873 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-6673178a-7f73-4740-9e99-f92ecba777d4 admin admin] The token cache is being invalidate because project 7df52edbfc594a60b8c175ef4a63875c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:29:27.092365 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-6673178a-7f73-4740-9e99-f92ecba777d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:27.093675 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-6673178a-7f73-4740-9e99-f92ecba777d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:27.096500 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1101/2222] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:29:26 2022] DELETE /identity/v3/projects/7df52edbfc594a60b8c175ef4a63875c => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:27.101209 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9516c6e1-5168-4d5c-9264-1ae8a2788a83 service neutron] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:27.125519 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9516c6e1-5168-4d5c-9264-1ae8a2788a83 service neutron] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:27.140950 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:27.242926 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95eb15d-f844-4038-9fb9-f26376bc2c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:27.246098 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95eb15d-f844-4038-9fb9-f26376bc2c58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:27.247228 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95eb15d-f844-4038-9fb9-f26376bc2c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:27.247936 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95eb15d-f844-4038-9fb9-f26376bc2c58 admin admin] PATH_INFO: `/v3/users/72defa94e12a43af9ea199cdc064d8c6` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:27.250897 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95eb15d-f844-4038-9fb9-f26376bc2c58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72defa94e12a43af9ea199cdc064d8c6)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:27.254760 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95eb15d-f844-4038-9fb9-f26376bc2c58 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:27.715599 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-c95eb15d-f844-4038-9fb9-f26376bc2c58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72defa94e12a43af9ea199cdc064d8c6'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:27.761127 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-c95eb15d-f844-4038-9fb9-f26376bc2c58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72defa94e12a43af9ea199cdc064d8c6'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:27.841054 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-c95eb15d-f844-4038-9fb9-f26376bc2c58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72defa94e12a43af9ea199cdc064d8c6'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:27.842140 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-c95eb15d-f844-4038-9fb9-f26376bc2c58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72defa94e12a43af9ea199cdc064d8c6'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:27.871448 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-c95eb15d-f844-4038-9fb9-f26376bc2c58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72defa94e12a43af9ea199cdc064d8c6'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:27.927563 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1122/2223] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:29:27 2022] DELETE /identity/v3/users/72defa94e12a43af9ea199cdc064d8c6 => generated 0 bytes in 827 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:28.775298 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6673178a-7f73-4740-9e99-f92ecba777d4 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:28.776999 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-6673178a-7f73-4740-9e99-f92ecba777d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:28.811107 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6673178a-7f73-4740-9e99-f92ecba777d4 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:28.939059 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed268123-4846-43fe-9003-91f6096096a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:28.940654 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed268123-4846-43fe-9003-91f6096096a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:28.941201 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed268123-4846-43fe-9003-91f6096096a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:28.941578 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed268123-4846-43fe-9003-91f6096096a4 admin admin] PATH_INFO: `/v3/projects/8d71f4e9434b4612a5afe4763ca28268` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:28.948811 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed268123-4846-43fe-9003-91f6096096a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d71f4e9434b4612a5afe4763ca28268)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:28.950514 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed268123-4846-43fe-9003-91f6096096a4 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:29.064685 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:29.086166 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-ed268123-4846-43fe-9003-91f6096096a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d71f4e9434b4612a5afe4763ca28268', 'request_id': 'req-ed268123-4846-43fe-9003-91f6096096a4'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:29.194720 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-ed268123-4846-43fe-9003-91f6096096a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d71f4e9434b4612a5afe4763ca28268', 'request_id': 'req-ed268123-4846-43fe-9003-91f6096096a4'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:29.203832 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-ed268123-4846-43fe-9003-91f6096096a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d71f4e9434b4612a5afe4763ca28268', 'request_id': 'req-ed268123-4846-43fe-9003-91f6096096a4'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:29.219540 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-ed268123-4846-43fe-9003-91f6096096a4 admin admin] The token cache is being invalidate because project 8d71f4e9434b4612a5afe4763ca28268 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:29:29.219833 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-ed268123-4846-43fe-9003-91f6096096a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:29.221380 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-ed268123-4846-43fe-9003-91f6096096a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:29.223405 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1102/2224] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:29:28 2022] DELETE /identity/v3/projects/8d71f4e9434b4612a5afe4763ca28268 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:29.231170 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7bd77d-531c-4070-8f45-964cca11391f None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:29.231170 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7bd77d-531c-4070-8f45-964cca11391f None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:29.231170 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7bd77d-531c-4070-8f45-964cca11391f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:29.282195 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-ec7bd77d-531c-4070-8f45-964cca11391f None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:29.284485 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-ec7bd77d-531c-4070-8f45-964cca11391f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:29.379128 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1123/2225] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:29:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:29.390074 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4517f8d-c2bc-4f5a-b0dd-5a5033e8d435 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:29.390896 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4517f8d-c2bc-4f5a-b0dd-5a5033e8d435 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:29.391358 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4517f8d-c2bc-4f5a-b0dd-5a5033e8d435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:29.413576 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-f4517f8d-c2bc-4f5a-b0dd-5a5033e8d435 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:29.415031 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-f4517f8d-c2bc-4f5a-b0dd-5a5033e8d435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:29.433841 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1103/2226] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:29:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:29:29.434541 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7bd77d-531c-4070-8f45-964cca11391f None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:29.444957 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7bd77d-531c-4070-8f45-964cca11391f None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:29.451127 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c457e9-07c8-4fb6-8d48-ada1bb4bf9d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:29.451942 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c457e9-07c8-4fb6-8d48-ada1bb4bf9d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:29.452371 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c457e9-07c8-4fb6-8d48-ada1bb4bf9d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:29.452770 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c457e9-07c8-4fb6-8d48-ada1bb4bf9d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:29.454000 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c457e9-07c8-4fb6-8d48-ada1bb4bf9d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:29:29.454348 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c457e9-07c8-4fb6-8d48-ada1bb4bf9d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:29.456266 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c457e9-07c8-4fb6-8d48-ada1bb4bf9d8 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:29.471019 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1124/2227] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:29:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:29:29.477666 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4517f8d-c2bc-4f5a-b0dd-5a5033e8d435 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:29.495281 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4517f8d-c2bc-4f5a-b0dd-5a5033e8d435 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:29.503918 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e57961-4c1e-400a-9ef4-0564f5c87fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:29.505057 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e57961-4c1e-400a-9ef4-0564f5c87fd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:29.505466 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e57961-4c1e-400a-9ef4-0564f5c87fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:29.505854 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e57961-4c1e-400a-9ef4-0564f5c87fd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:29.511909 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e57961-4c1e-400a-9ef4-0564f5c87fd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:29.512152 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e57961-4c1e-400a-9ef4-0564f5c87fd8 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:29.668032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1104/2228] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:29:29 2022] POST /identity/v3/projects => generated 361 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:29:29.675341 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c457e9-07c8-4fb6-8d48-ada1bb4bf9d8 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:29.690374 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c457e9-07c8-4fb6-8d48-ada1bb4bf9d8 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:29.704458 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:29.778568 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350845c2-e3c1-4814-bbd6-f7fc738fd4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:29.779522 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350845c2-e3c1-4814-bbd6-f7fc738fd4c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:29.779964 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350845c2-e3c1-4814-bbd6-f7fc738fd4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:29.780249 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350845c2-e3c1-4814-bbd6-f7fc738fd4c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:29.781136 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350845c2-e3c1-4814-bbd6-f7fc738fd4c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:29.782999 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350845c2-e3c1-4814-bbd6-f7fc738fd4c5 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:29.912490 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1125/2229] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:29:29 2022] POST /identity/v3/users => generated 281 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:29:29.916856 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e57961-4c1e-400a-9ef4-0564f5c87fd8 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:29.929439 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e57961-4c1e-400a-9ef4-0564f5c87fd8 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:29.937240 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d500aa4-7e23-4179-96c4-7b671facc239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:29.938481 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d500aa4-7e23-4179-96c4-7b671facc239 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:29.938915 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d500aa4-7e23-4179-96c4-7b671facc239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:29.939346 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d500aa4-7e23-4179-96c4-7b671facc239 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:29.940642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d500aa4-7e23-4179-96c4-7b671facc239 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:29.944506 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d500aa4-7e23-4179-96c4-7b671facc239 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:29.967789 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1105/2230] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:29 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:29:29.971910 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350845c2-e3c1-4814-bbd6-f7fc738fd4c5 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:29.989939 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350845c2-e3c1-4814-bbd6-f7fc738fd4c5 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:29.998729 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baa99c5-ae88-4d20-b20a-ed27804037ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:30.000506 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa99c5-ae88-4d20-b20a-ed27804037ae admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:30.001290 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa99c5-ae88-4d20-b20a-ed27804037ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:30.002093 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa99c5-ae88-4d20-b20a-ed27804037ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:30.003218 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baa99c5-ae88-4d20-b20a-ed27804037ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:30.006072 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baa99c5-ae88-4d20-b20a-ed27804037ae admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:30.032730 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1126/2231] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:29 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:29:30.034247 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d500aa4-7e23-4179-96c4-7b671facc239 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:30.050567 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d500aa4-7e23-4179-96c4-7b671facc239 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:30.056260 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120ec9ea-eb90-454a-a0c3-574a3025fe92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:30.057550 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120ec9ea-eb90-454a-a0c3-574a3025fe92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:30.058053 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120ec9ea-eb90-454a-a0c3-574a3025fe92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:30.058582 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120ec9ea-eb90-454a-a0c3-574a3025fe92 admin admin] PATH_INFO: `/v3/projects/098b56da8c20403eb8e313a4648ae2d5/users/25b4288e56544220ae319cbdf5a43ac2/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:30.090799 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120ec9ea-eb90-454a-a0c3-574a3025fe92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=098b56da8c20403eb8e313a4648ae2d5, user_id=25b4288e56544220ae319cbdf5a43ac2, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:30.099502 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120ec9ea-eb90-454a-a0c3-574a3025fe92 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:30.175916 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1106/2232] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:29:30 2022] PUT /identity/v3/projects/098b56da8c20403eb8e313a4648ae2d5/users/25b4288e56544220ae319cbdf5a43ac2/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:30.187303 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77848033-07d7-4655-80fe-c37248616b6f None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:30.187812 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77848033-07d7-4655-80fe-c37248616b6f None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:30.188228 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77848033-07d7-4655-80fe-c37248616b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:30.222179 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-77848033-07d7-4655-80fe-c37248616b6f None None] MFA Rules not processed for user `25b4288e56544220ae319cbdf5a43ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:30.223485 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-77848033-07d7-4655-80fe-c37248616b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:30.305361 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1127/2233] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:29:30 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:30.329721 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120ec9ea-eb90-454a-a0c3-574a3025fe92 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:30.333489 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-120ec9ea-eb90-454a-a0c3-574a3025fe92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:30.371886 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77848033-07d7-4655-80fe-c37248616b6f None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:30.374032 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-77848033-07d7-4655-80fe-c37248616b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:30.432734 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120ec9ea-eb90-454a-a0c3-574a3025fe92 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:30.440940 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f1c5f7-2ec4-4479-9cb9-6c3408a62501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:30.442498 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f1c5f7-2ec4-4479-9cb9-6c3408a62501 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:30.443232 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f1c5f7-2ec4-4479-9cb9-6c3408a62501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:30.443713 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f1c5f7-2ec4-4479-9cb9-6c3408a62501 admin admin] PATH_INFO: `/v3/users/6630f77b1dba4187a267773246bd2194` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:30.447507 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f1c5f7-2ec4-4479-9cb9-6c3408a62501 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6630f77b1dba4187a267773246bd2194)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:30.451267 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f1c5f7-2ec4-4479-9cb9-6c3408a62501 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:30.499236 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77848033-07d7-4655-80fe-c37248616b6f None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:30.503749 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9abdb70-d10e-414e-968b-63e962adf40a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:30.504665 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9abdb70-d10e-414e-968b-63e962adf40a service cinder] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:30.505019 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9abdb70-d10e-414e-968b-63e962adf40a service cinder] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:30.505370 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9abdb70-d10e-414e-968b-63e962adf40a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:30.516510 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9abdb70-d10e-414e-968b-63e962adf40a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:30.518137 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9abdb70-d10e-414e-968b-63e962adf40a service cinder] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:30.530874 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1128/2234] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:29:30 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:29:30.547863 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9abdb70-d10e-414e-968b-63e962adf40a service cinder] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:30.565132 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9abdb70-d10e-414e-968b-63e962adf40a service cinder] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:30.574124 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1135dc8-e0a8-48bc-8eee-5f1c7045be51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:30.575521 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1135dc8-e0a8-48bc-8eee-5f1c7045be51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:30.576094 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1135dc8-e0a8-48bc-8eee-5f1c7045be51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:30.576529 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1135dc8-e0a8-48bc-8eee-5f1c7045be51 admin admin] PATH_INFO: `/v3/users/25b4288e56544220ae319cbdf5a43ac2` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:30.578230 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1135dc8-e0a8-48bc-8eee-5f1c7045be51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25b4288e56544220ae319cbdf5a43ac2)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:30.580608 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1135dc8-e0a8-48bc-8eee-5f1c7045be51 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:30.614933 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:30.674355 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-d1f1c5f7-2ec4-4479-9cb9-6c3408a62501 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6630f77b1dba4187a267773246bd2194'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:30.700592 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:30.700592 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-d1f1c5f7-2ec4-4479-9cb9-6c3408a62501 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6630f77b1dba4187a267773246bd2194'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:30.700592 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-d1f1c5f7-2ec4-4479-9cb9-6c3408a62501 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6630f77b1dba4187a267773246bd2194'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:30.708360 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-d1f1c5f7-2ec4-4479-9cb9-6c3408a62501 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6630f77b1dba4187a267773246bd2194'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:30.709151 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-d1f1c5f7-2ec4-4479-9cb9-6c3408a62501 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6630f77b1dba4187a267773246bd2194'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:30.915139 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1107/2235] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:29:30 2022] DELETE /identity/v3/users/6630f77b1dba4187a267773246bd2194 => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:30.924896 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a3dda-44a7-401b-9482-42d776bbc544 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:30.925929 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a3dda-44a7-401b-9482-42d776bbc544 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:30.925929 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a3dda-44a7-401b-9482-42d776bbc544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:30.949397 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-e1135dc8-e0a8-48bc-8eee-5f1c7045be51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25b4288e56544220ae319cbdf5a43ac2'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:30.959541 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-e1135dc8-e0a8-48bc-8eee-5f1c7045be51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25b4288e56544220ae319cbdf5a43ac2'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:30.965239 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-9b4a3dda-44a7-401b-9482-42d776bbc544 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:30.968050 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-9b4a3dda-44a7-401b-9482-42d776bbc544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:31.035885 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-e1135dc8-e0a8-48bc-8eee-5f1c7045be51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25b4288e56544220ae319cbdf5a43ac2'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:31.036740 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-e1135dc8-e0a8-48bc-8eee-5f1c7045be51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25b4288e56544220ae319cbdf5a43ac2'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:31.052377 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-e1135dc8-e0a8-48bc-8eee-5f1c7045be51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25b4288e56544220ae319cbdf5a43ac2'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:31.089228 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1108/2236] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:29:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:31.089482 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1129/2237] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:29:30 2022] DELETE /identity/v3/users/25b4288e56544220ae319cbdf5a43ac2 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:29:31.091860 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c45a07-9a1f-47b6-b294-3b3c4c4076ef None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:31.093358 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c45a07-9a1f-47b6-b294-3b3c4c4076ef None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:31.093672 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c45a07-9a1f-47b6-b294-3b3c4c4076ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:31.102068 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1135dc8-e0a8-48bc-8eee-5f1c7045be51 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:31.109206 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-e1135dc8-e0a8-48bc-8eee-5f1c7045be51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:31.157666 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-95c45a07-9a1f-47b6-b294-3b3c4c4076ef None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:31.160712 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-95c45a07-9a1f-47b6-b294-3b3c4c4076ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:31.298020 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1135dc8-e0a8-48bc-8eee-5f1c7045be51 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:31.305849 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a31344-7055-4526-8d8d-e8fedf3c19f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:31.307947 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a31344-7055-4526-8d8d-e8fedf3c19f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:31.307947 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a31344-7055-4526-8d8d-e8fedf3c19f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:31.307947 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a31344-7055-4526-8d8d-e8fedf3c19f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:31.336839 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a31344-7055-4526-8d8d-e8fedf3c19f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:31.336839 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a31344-7055-4526-8d8d-e8fedf3c19f0 service neutron] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:31.342745 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1109/2238] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:29:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:31.347942 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c45a07-9a1f-47b6-b294-3b3c4c4076ef None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:31.348234 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1130/2239] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:29:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:29:31.370991 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c45a07-9a1f-47b6-b294-3b3c4c4076ef None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:31.378095 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f76da44-4f34-46f9-9fc3-871c948b2e7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:31.381943 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76da44-4f34-46f9-9fc3-871c948b2e7d service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:31.381943 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76da44-4f34-46f9-9fc3-871c948b2e7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:31.381943 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76da44-4f34-46f9-9fc3-871c948b2e7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:31.410058 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f76da44-4f34-46f9-9fc3-871c948b2e7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:31.413262 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f76da44-4f34-46f9-9fc3-871c948b2e7d service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:31.427167 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1110/2240] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:29:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:29:32.212079 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a31344-7055-4526-8d8d-e8fedf3c19f0 service neutron] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:32.215031 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-b9a31344-7055-4526-8d8d-e8fedf3c19f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:32.242216 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a31344-7055-4526-8d8d-e8fedf3c19f0 service neutron] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:32.252250 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5c1869-c3e9-4ca3-9c53-b23979bae204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:32.254409 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c1869-c3e9-4ca3-9c53-b23979bae204 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:32.255183 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c1869-c3e9-4ca3-9c53-b23979bae204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:32.255905 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c1869-c3e9-4ca3-9c53-b23979bae204 admin admin] PATH_INFO: `/v3/projects/ce6fadeffb9442fb83b5f1162d71e91e` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:32.258911 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5c1869-c3e9-4ca3-9c53-b23979bae204 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce6fadeffb9442fb83b5f1162d71e91e)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:32.263935 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5c1869-c3e9-4ca3-9c53-b23979bae204 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:32.453812 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-2e5c1869-c3e9-4ca3-9c53-b23979bae204 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce6fadeffb9442fb83b5f1162d71e91e', 'request_id': 'req-2e5c1869-c3e9-4ca3-9c53-b23979bae204'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:32.473276 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-2e5c1869-c3e9-4ca3-9c53-b23979bae204 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce6fadeffb9442fb83b5f1162d71e91e', 'request_id': 'req-2e5c1869-c3e9-4ca3-9c53-b23979bae204'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:32.483362 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-2e5c1869-c3e9-4ca3-9c53-b23979bae204 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce6fadeffb9442fb83b5f1162d71e91e', 'request_id': 'req-2e5c1869-c3e9-4ca3-9c53-b23979bae204'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:32.598607 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-2e5c1869-c3e9-4ca3-9c53-b23979bae204 admin admin] The token cache is being invalidate because project ce6fadeffb9442fb83b5f1162d71e91e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:29:32.599135 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-2e5c1869-c3e9-4ca3-9c53-b23979bae204 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:32.600119 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-2e5c1869-c3e9-4ca3-9c53-b23979bae204 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:32.602773 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1131/2241] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:29:32 2022] DELETE /identity/v3/projects/ce6fadeffb9442fb83b5f1162d71e91e => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:32.714194 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f76da44-4f34-46f9-9fc3-871c948b2e7d service neutron] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:32.725168 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-0f76da44-4f34-46f9-9fc3-871c948b2e7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:32.745606 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f76da44-4f34-46f9-9fc3-871c948b2e7d service neutron] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:32.899023 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:32.917402 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103f2d55-50b9-4ec0-9c7c-fc9b3b1cb038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:32.919764 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f2d55-50b9-4ec0-9c7c-fc9b3b1cb038 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:32.920318 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f2d55-50b9-4ec0-9c7c-fc9b3b1cb038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:32.920994 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f2d55-50b9-4ec0-9c7c-fc9b3b1cb038 admin admin] PATH_INFO: `/v3/projects/098b56da8c20403eb8e313a4648ae2d5` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:32.924564 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103f2d55-50b9-4ec0-9c7c-fc9b3b1cb038 admin admin] RBAC: Authorizing `identity:delete_project(project_id=098b56da8c20403eb8e313a4648ae2d5)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:32.928684 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103f2d55-50b9-4ec0-9c7c-fc9b3b1cb038 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:33.039810 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5c1869-c3e9-4ca3-9c53-b23979bae204 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:33.043067 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-2e5c1869-c3e9-4ca3-9c53-b23979bae204 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:33.076309 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-103f2d55-50b9-4ec0-9c7c-fc9b3b1cb038 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '098b56da8c20403eb8e313a4648ae2d5', 'request_id': 'req-103f2d55-50b9-4ec0-9c7c-fc9b3b1cb038'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:33.081813 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5c1869-c3e9-4ca3-9c53-b23979bae204 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:33.139052 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:33.183859 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-103f2d55-50b9-4ec0-9c7c-fc9b3b1cb038 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '098b56da8c20403eb8e313a4648ae2d5', 'request_id': 'req-103f2d55-50b9-4ec0-9c7c-fc9b3b1cb038'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:33.192642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-103f2d55-50b9-4ec0-9c7c-fc9b3b1cb038 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '098b56da8c20403eb8e313a4648ae2d5', 'request_id': 'req-103f2d55-50b9-4ec0-9c7c-fc9b3b1cb038'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:33.227684 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a167a16-3cd2-4d00-98ee-4be5cb3dae6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:33.230103 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a167a16-3cd2-4d00-98ee-4be5cb3dae6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:33.230603 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a167a16-3cd2-4d00-98ee-4be5cb3dae6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:33.231210 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a167a16-3cd2-4d00-98ee-4be5cb3dae6d admin admin] PATH_INFO: `/v3/users/7dc90a9a937e4414b373982c96027a67` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:33.235907 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a167a16-3cd2-4d00-98ee-4be5cb3dae6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7dc90a9a937e4414b373982c96027a67)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:33.236142 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-103f2d55-50b9-4ec0-9c7c-fc9b3b1cb038 admin admin] The token cache is being invalidate because project 098b56da8c20403eb8e313a4648ae2d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:29:33.236520 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-103f2d55-50b9-4ec0-9c7c-fc9b3b1cb038 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:33.237564 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-103f2d55-50b9-4ec0-9c7c-fc9b3b1cb038 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:33.241683 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a167a16-3cd2-4d00-98ee-4be5cb3dae6d admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:33.246418 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1111/2242] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:29:32 2022] DELETE /identity/v3/projects/098b56da8c20403eb8e313a4648ae2d5 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:33.248921 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14240e08-d497-418f-b8be-8e3ec98dd165 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:33.249413 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14240e08-d497-418f-b8be-8e3ec98dd165 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:33.250153 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14240e08-d497-418f-b8be-8e3ec98dd165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:33.293750 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-14240e08-d497-418f-b8be-8e3ec98dd165 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:33.297166 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-14240e08-d497-418f-b8be-8e3ec98dd165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:33.326353 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1112/2243] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:29:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:33.337655 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76c71bd-be44-41dd-bd42-7a1c5ff30e3b None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:33.338118 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76c71bd-be44-41dd-bd42-7a1c5ff30e3b None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:33.338981 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76c71bd-be44-41dd-bd42-7a1c5ff30e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:33.361662 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-c76c71bd-be44-41dd-bd42-7a1c5ff30e3b None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:33.371063 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-c76c71bd-be44-41dd-bd42-7a1c5ff30e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:33.376904 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1113/2244] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:29:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:29:33.384132 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76c71bd-be44-41dd-bd42-7a1c5ff30e3b None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:33.404834 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76c71bd-be44-41dd-bd42-7a1c5ff30e3b None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:33.413300 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d403b29-69a2-4900-9f3b-cb0bc5fa0d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:33.414837 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d403b29-69a2-4900-9f3b-cb0bc5fa0d18 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:33.415419 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d403b29-69a2-4900-9f3b-cb0bc5fa0d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:33.416585 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d403b29-69a2-4900-9f3b-cb0bc5fa0d18 admin admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:33.418125 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d403b29-69a2-4900-9f3b-cb0bc5fa0d18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:29:33.418707 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d403b29-69a2-4900-9f3b-cb0bc5fa0d18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:33.422057 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d403b29-69a2-4900-9f3b-cb0bc5fa0d18 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:33.452453 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1114/2245] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:29:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:29:33.456173 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d403b29-69a2-4900-9f3b-cb0bc5fa0d18 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:33.470172 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d403b29-69a2-4900-9f3b-cb0bc5fa0d18 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:33.476678 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df56e85-d100-4ab8-a11d-82530c2b1f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:33.477361 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df56e85-d100-4ab8-a11d-82530c2b1f08 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:33.477695 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df56e85-d100-4ab8-a11d-82530c2b1f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:33.477979 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df56e85-d100-4ab8-a11d-82530c2b1f08 admin admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:33.479312 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df56e85-d100-4ab8-a11d-82530c2b1f08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:33.481301 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df56e85-d100-4ab8-a11d-82530c2b1f08 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:33.572752 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-2a167a16-3cd2-4d00-98ee-4be5cb3dae6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7dc90a9a937e4414b373982c96027a67'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:33.588717 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-2a167a16-3cd2-4d00-98ee-4be5cb3dae6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7dc90a9a937e4414b373982c96027a67'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:33.599264 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1115/2246] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:29:33 2022] POST /identity/v3/projects => generated 389 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:29:33.604250 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df56e85-d100-4ab8-a11d-82530c2b1f08 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:33.680362 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df56e85-d100-4ab8-a11d-82530c2b1f08 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:33.686224 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222717f0-b580-4aff-9b1c-559610bb1a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:33.687058 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222717f0-b580-4aff-9b1c-559610bb1a2a admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:33.687360 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222717f0-b580-4aff-9b1c-559610bb1a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:33.687640 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222717f0-b580-4aff-9b1c-559610bb1a2a admin admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:33.688374 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222717f0-b580-4aff-9b1c-559610bb1a2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:33.690485 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222717f0-b580-4aff-9b1c-559610bb1a2a admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:33.708120 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-2a167a16-3cd2-4d00-98ee-4be5cb3dae6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7dc90a9a937e4414b373982c96027a67'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:33.708801 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-2a167a16-3cd2-4d00-98ee-4be5cb3dae6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7dc90a9a937e4414b373982c96027a67'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:33.717973 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-2a167a16-3cd2-4d00-98ee-4be5cb3dae6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7dc90a9a937e4414b373982c96027a67'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:33.725052 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1132/2247] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:29:33 2022] DELETE /identity/v3/users/7dc90a9a937e4414b373982c96027a67 => generated 0 bytes in 686 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:33.814442 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1116/2248] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:29:33 2022] POST /identity/v3/users => generated 295 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:29:33.820614 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a167a16-3cd2-4d00-98ee-4be5cb3dae6d admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:33.824010 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-2a167a16-3cd2-4d00-98ee-4be5cb3dae6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:33.959860 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a167a16-3cd2-4d00-98ee-4be5cb3dae6d admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:33.970007 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfdc894-6020-4f1e-b4fe-d2b36f22b01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:33.971084 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfdc894-6020-4f1e-b4fe-d2b36f22b01c admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:33.972243 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfdc894-6020-4f1e-b4fe-d2b36f22b01c admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:33.972243 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfdc894-6020-4f1e-b4fe-d2b36f22b01c admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:33.973752 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfdc894-6020-4f1e-b4fe-d2b36f22b01c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:33.977689 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfdc894-6020-4f1e-b4fe-d2b36f22b01c admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:33.997599 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1133/2249] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:33 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:29:34.002714 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222717f0-b580-4aff-9b1c-559610bb1a2a admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:34.022292 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222717f0-b580-4aff-9b1c-559610bb1a2a admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:34.030250 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc11970-5032-40bd-a766-709f1cb8a4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:34.031334 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc11970-5032-40bd-a766-709f1cb8a4de admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:34.031987 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc11970-5032-40bd-a766-709f1cb8a4de admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:34.032448 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc11970-5032-40bd-a766-709f1cb8a4de admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:34.033542 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc11970-5032-40bd-a766-709f1cb8a4de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:34.035690 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc11970-5032-40bd-a766-709f1cb8a4de admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:34.052110 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1117/2250] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:29:34.058183 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfdc894-6020-4f1e-b4fe-d2b36f22b01c admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:34.072999 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfdc894-6020-4f1e-b4fe-d2b36f22b01c admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:34.080430 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f0398d-ce27-430e-92fb-6819c806af71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:34.082358 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0398d-ce27-430e-92fb-6819c806af71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:34.082358 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0398d-ce27-430e-92fb-6819c806af71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:34.082358 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0398d-ce27-430e-92fb-6819c806af71 admin admin] PATH_INFO: `/v3/projects/b9b64f5e855745b4800958294d93315e/users/b8521e4b29434f04b51b9bcabab0e5b1/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:34.104426 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f0398d-ce27-430e-92fb-6819c806af71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9b64f5e855745b4800958294d93315e, user_id=b8521e4b29434f04b51b9bcabab0e5b1, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:34.107399 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f0398d-ce27-430e-92fb-6819c806af71 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:34.135201 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc11970-5032-40bd-a766-709f1cb8a4de admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:34.137427 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-9cc11970-5032-40bd-a766-709f1cb8a4de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:34.156045 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc11970-5032-40bd-a766-709f1cb8a4de admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:34.164648 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deb2655-5360-4670-84b8-feed500c2532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:34.166291 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb2655-5360-4670-84b8-feed500c2532 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:34.166792 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb2655-5360-4670-84b8-feed500c2532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:34.167288 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb2655-5360-4670-84b8-feed500c2532 admin admin] PATH_INFO: `/v3/projects/02f7adbad51e4f29aae3da01ec334872` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:34.169193 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deb2655-5360-4670-84b8-feed500c2532 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02f7adbad51e4f29aae3da01ec334872)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:34.172992 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deb2655-5360-4670-84b8-feed500c2532 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:34.192755 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:34.193630 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1134/2251] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:29:34 2022] PUT /identity/v3/projects/b9b64f5e855745b4800958294d93315e/users/b8521e4b29434f04b51b9bcabab0e5b1/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:29:34.222948 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8fb4b-3c64-4efd-b33e-6d84d07948f2 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:34.222948 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8fb4b-3c64-4efd-b33e-6d84d07948f2 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:34.222948 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8fb4b-3c64-4efd-b33e-6d84d07948f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:34.251236 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:34.263617 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-f9e8fb4b-3c64-4efd-b33e-6d84d07948f2 None None] MFA Rules not processed for user `b8521e4b29434f04b51b9bcabab0e5b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:34.265139 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-f9e8fb4b-3c64-4efd-b33e-6d84d07948f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:34.344677 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-2deb2655-5360-4670-84b8-feed500c2532 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02f7adbad51e4f29aae3da01ec334872', 'request_id': 'req-2deb2655-5360-4670-84b8-feed500c2532'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:34.349899 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1135/2252] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:29:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:34.363895 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e8fb4b-3c64-4efd-b33e-6d84d07948f2 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:34.365378 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-f9e8fb4b-3c64-4efd-b33e-6d84d07948f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:34.428042 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-2deb2655-5360-4670-84b8-feed500c2532 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02f7adbad51e4f29aae3da01ec334872', 'request_id': 'req-2deb2655-5360-4670-84b8-feed500c2532'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:34.437981 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-2deb2655-5360-4670-84b8-feed500c2532 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02f7adbad51e4f29aae3da01ec334872', 'request_id': 'req-2deb2655-5360-4670-84b8-feed500c2532'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:34.453020 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e8fb4b-3c64-4efd-b33e-6d84d07948f2 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:34.460123 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-2deb2655-5360-4670-84b8-feed500c2532 admin admin] The token cache is being invalidate because project 02f7adbad51e4f29aae3da01ec334872 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:29:34.460834 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-2deb2655-5360-4670-84b8-feed500c2532 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:34.462946 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-2deb2655-5360-4670-84b8-feed500c2532 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:34.466533 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1118/2253] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:29:34 2022] DELETE /identity/v3/projects/02f7adbad51e4f29aae3da01ec334872 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:29:34.480614 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deb2655-5360-4670-84b8-feed500c2532 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:34.480614 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-2deb2655-5360-4670-84b8-feed500c2532 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:34.559858 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d1e9cd-b1ba-44bf-9abb-038b399529f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:34.561017 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d1e9cd-b1ba-44bf-9abb-038b399529f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:34.561478 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d1e9cd-b1ba-44bf-9abb-038b399529f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:34.562294 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d1e9cd-b1ba-44bf-9abb-038b399529f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:34.565020 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deb2655-5360-4670-84b8-feed500c2532 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:34.566159 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-23d1e9cd-b1ba-44bf-9abb-038b399529f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:34.570833 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2efa728-afa5-4a90-8e36-3c903f0e4d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:34.572868 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2efa728-afa5-4a90-8e36-3c903f0e4d42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:34.573377 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2efa728-afa5-4a90-8e36-3c903f0e4d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:34.573799 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2efa728-afa5-4a90-8e36-3c903f0e4d42 admin admin] PATH_INFO: `/v3/users/767a9b4eb6ba4237b12b1e2dc0702af1` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:34.575558 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2efa728-afa5-4a90-8e36-3c903f0e4d42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=767a9b4eb6ba4237b12b1e2dc0702af1)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:34.578247 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2efa728-afa5-4a90-8e36-3c903f0e4d42 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:34.648948 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d1e9cd-b1ba-44bf-9abb-038b399529f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:34.651198 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d1e9cd-b1ba-44bf-9abb-038b399529f1 service cinder] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:34.662018 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1136/2254] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:29:34 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:29:34.926394 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-f2efa728-afa5-4a90-8e36-3c903f0e4d42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '767a9b4eb6ba4237b12b1e2dc0702af1'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:34.950759 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-f2efa728-afa5-4a90-8e36-3c903f0e4d42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '767a9b4eb6ba4237b12b1e2dc0702af1'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:34.951659 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-f2efa728-afa5-4a90-8e36-3c903f0e4d42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '767a9b4eb6ba4237b12b1e2dc0702af1'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:34.960741 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-f2efa728-afa5-4a90-8e36-3c903f0e4d42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '767a9b4eb6ba4237b12b1e2dc0702af1'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:34.961598 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-f2efa728-afa5-4a90-8e36-3c903f0e4d42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '767a9b4eb6ba4237b12b1e2dc0702af1'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:35.062799 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1119/2255] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:29:34 2022] DELETE /identity/v3/users/767a9b4eb6ba4237b12b1e2dc0702af1 => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:35.398945 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d1e9cd-b1ba-44bf-9abb-038b399529f1 service cinder] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:35.412063 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d1e9cd-b1ba-44bf-9abb-038b399529f1 service cinder] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:35.428852 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:35.511417 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b849950-4b5f-47d3-8f20-df1f62048b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:35.512669 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b849950-4b5f-47d3-8f20-df1f62048b68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:35.513103 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b849950-4b5f-47d3-8f20-df1f62048b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:35.513686 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b849950-4b5f-47d3-8f20-df1f62048b68 admin admin] PATH_INFO: `/v3/projects/50a09afb7faf46f19bea95e1f6cb080e` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:35.515319 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b849950-4b5f-47d3-8f20-df1f62048b68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50a09afb7faf46f19bea95e1f6cb080e)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:35.519787 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b849950-4b5f-47d3-8f20-df1f62048b68 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:35.715207 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-7b849950-4b5f-47d3-8f20-df1f62048b68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50a09afb7faf46f19bea95e1f6cb080e', 'request_id': 'req-7b849950-4b5f-47d3-8f20-df1f62048b68'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:35.726999 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-7b849950-4b5f-47d3-8f20-df1f62048b68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50a09afb7faf46f19bea95e1f6cb080e', 'request_id': 'req-7b849950-4b5f-47d3-8f20-df1f62048b68'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:35.744336 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-7b849950-4b5f-47d3-8f20-df1f62048b68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50a09afb7faf46f19bea95e1f6cb080e', 'request_id': 'req-7b849950-4b5f-47d3-8f20-df1f62048b68'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:35.847482 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-7b849950-4b5f-47d3-8f20-df1f62048b68 admin admin] The token cache is being invalidate because project 50a09afb7faf46f19bea95e1f6cb080e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:29:35.848072 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-7b849950-4b5f-47d3-8f20-df1f62048b68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:35.849199 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-7b849950-4b5f-47d3-8f20-df1f62048b68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:35.851946 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1137/2256] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:29:35 2022] DELETE /identity/v3/projects/50a09afb7faf46f19bea95e1f6cb080e => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:35.860088 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf31f92-d0c7-46bd-af20-c5bfecda3ce2 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:35.860569 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf31f92-d0c7-46bd-af20-c5bfecda3ce2 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:35.861198 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf31f92-d0c7-46bd-af20-c5bfecda3ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:35.886621 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-0bf31f92-d0c7-46bd-af20-c5bfecda3ce2 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:35.888527 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-0bf31f92-d0c7-46bd-af20-c5bfecda3ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:35.908376 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:35.966550 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1120/2257] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:29:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:35.976507 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16709fa7-d9b0-4996-9704-d27ce07d1e13 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:35.976861 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16709fa7-d9b0-4996-9704-d27ce07d1e13 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:35.977189 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16709fa7-d9b0-4996-9704-d27ce07d1e13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:35.999040 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-16709fa7-d9b0-4996-9704-d27ce07d1e13 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:36.000488 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-16709fa7-d9b0-4996-9704-d27ce07d1e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:36.010068 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1138/2258] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:29:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:36.017095 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf31f92-d0c7-46bd-af20-c5bfecda3ce2 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:36.028947 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf31f92-d0c7-46bd-af20-c5bfecda3ce2 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:36.034203 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4a45e-0be2-4851-af19-65d7d510db2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:36.034931 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4a45e-0be2-4851-af19-65d7d510db2d admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:36.037253 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4a45e-0be2-4851-af19-65d7d510db2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:36.037253 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4a45e-0be2-4851-af19-65d7d510db2d admin admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:36.037253 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d4a45e-0be2-4851-af19-65d7d510db2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:29:36.037253 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d4a45e-0be2-4851-af19-65d7d510db2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:36.038517 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d4a45e-0be2-4851-af19-65d7d510db2d admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:36.055432 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1121/2259] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:29:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:29:36.061924 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16709fa7-d9b0-4996-9704-d27ce07d1e13 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:36.075052 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16709fa7-d9b0-4996-9704-d27ce07d1e13 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:36.081360 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea59657-67ed-4a8c-b107-1ac13b0c1588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:36.082269 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea59657-67ed-4a8c-b107-1ac13b0c1588 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:36.082652 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea59657-67ed-4a8c-b107-1ac13b0c1588 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:36.083199 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea59657-67ed-4a8c-b107-1ac13b0c1588 admin admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:36.084023 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea59657-67ed-4a8c-b107-1ac13b0c1588 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:36.086241 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea59657-67ed-4a8c-b107-1ac13b0c1588 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:36.250097 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1139/2260] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:29:36 2022] POST /identity/v3/projects => generated 371 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:29:36.256739 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4a45e-0be2-4851-af19-65d7d510db2d admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:36.277131 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4a45e-0be2-4851-af19-65d7d510db2d admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:36.387436 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fdc376-13e6-4e21-ae46-e384ca48d854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:36.388606 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fdc376-13e6-4e21-ae46-e384ca48d854 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:36.389237 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fdc376-13e6-4e21-ae46-e384ca48d854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:36.389802 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fdc376-13e6-4e21-ae46-e384ca48d854 admin admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:36.395892 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fdc376-13e6-4e21-ae46-e384ca48d854 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:36.395892 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fdc376-13e6-4e21-ae46-e384ca48d854 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:36.512542 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1122/2261] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:29:36 2022] POST /identity/v3/users => generated 286 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:29:36.517371 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea59657-67ed-4a8c-b107-1ac13b0c1588 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:36.529346 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea59657-67ed-4a8c-b107-1ac13b0c1588 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:36.534420 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016128ac-2e99-4496-b1db-921e399196c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:36.535147 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016128ac-2e99-4496-b1db-921e399196c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:36.535378 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016128ac-2e99-4496-b1db-921e399196c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:36.535647 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016128ac-2e99-4496-b1db-921e399196c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:36.536420 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016128ac-2e99-4496-b1db-921e399196c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:36.538048 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016128ac-2e99-4496-b1db-921e399196c6 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:36.551322 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1140/2262] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:29:36.557814 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fdc376-13e6-4e21-ae46-e384ca48d854 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:36.568852 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fdc376-13e6-4e21-ae46-e384ca48d854 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:36.574634 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aeb1b0-a7b5-4025-81f9-614d58cd3124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:36.575530 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aeb1b0-a7b5-4025-81f9-614d58cd3124 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:36.575822 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aeb1b0-a7b5-4025-81f9-614d58cd3124 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:36.576105 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aeb1b0-a7b5-4025-81f9-614d58cd3124 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:36.576894 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aeb1b0-a7b5-4025-81f9-614d58cd3124 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:36.578551 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aeb1b0-a7b5-4025-81f9-614d58cd3124 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:36.595579 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1123/2263] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:29:36.596972 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016128ac-2e99-4496-b1db-921e399196c6 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:36.610785 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016128ac-2e99-4496-b1db-921e399196c6 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:36.615849 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596dba5a-e25e-4858-875f-89a93547542f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:36.617169 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596dba5a-e25e-4858-875f-89a93547542f admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:36.617472 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596dba5a-e25e-4858-875f-89a93547542f admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:36.617783 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596dba5a-e25e-4858-875f-89a93547542f admin admin] PATH_INFO: `/v3/projects/c2efa42f13bd40e58fc33c2f17155631/users/e33ee5e7c3a84032a521f7981148fbf1/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:36.640084 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596dba5a-e25e-4858-875f-89a93547542f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2efa42f13bd40e58fc33c2f17155631, user_id=e33ee5e7c3a84032a521f7981148fbf1, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:36.642913 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596dba5a-e25e-4858-875f-89a93547542f admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:36.772440 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1141/2264] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:29:36 2022] PUT /identity/v3/projects/c2efa42f13bd40e58fc33c2f17155631/users/e33ee5e7c3a84032a521f7981148fbf1/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:36.783150 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927e7741-0986-40e0-9105-80f1519ac54c None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:36.783723 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927e7741-0986-40e0-9105-80f1519ac54c None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:36.784030 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927e7741-0986-40e0-9105-80f1519ac54c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:36.816730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-927e7741-0986-40e0-9105-80f1519ac54c None None] MFA Rules not processed for user `e33ee5e7c3a84032a521f7981148fbf1`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:36.817991 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-927e7741-0986-40e0-9105-80f1519ac54c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:36.897853 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1124/2265] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:29:36 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:29:36.910709 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596dba5a-e25e-4858-875f-89a93547542f admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:36.913115 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-596dba5a-e25e-4858-875f-89a93547542f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:36.966883 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:36.986139 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596dba5a-e25e-4858-875f-89a93547542f admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:36.990053 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a9e46e-09c2-4f31-9387-b40ff3d6f335 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:36.990785 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a9e46e-09c2-4f31-9387-b40ff3d6f335 service cinder] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:36.999376 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a9e46e-09c2-4f31-9387-b40ff3d6f335 service cinder] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:36.999376 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a9e46e-09c2-4f31-9387-b40ff3d6f335 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:37.001595 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a9e46e-09c2-4f31-9387-b40ff3d6f335 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:37.003078 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a9e46e-09c2-4f31-9387-b40ff3d6f335 service cinder] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:37.014367 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1142/2266] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:29:36 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:29:37.042144 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927e7741-0986-40e0-9105-80f1519ac54c None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:37.063783 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:37.156110 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927e7741-0986-40e0-9105-80f1519ac54c None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:37.161500 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d5114-6eb1-44a9-a8e4-8bc280a1e857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:37.162445 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d5114-6eb1-44a9-a8e4-8bc280a1e857 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:37.162732 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d5114-6eb1-44a9-a8e4-8bc280a1e857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:37.163010 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d5114-6eb1-44a9-a8e4-8bc280a1e857 admin admin] PATH_INFO: `/v3/users/e33ee5e7c3a84032a521f7981148fbf1` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:37.164488 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00d5114-6eb1-44a9-a8e4-8bc280a1e857 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e33ee5e7c3a84032a521f7981148fbf1)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:37.166609 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00d5114-6eb1-44a9-a8e4-8bc280a1e857 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:37.238967 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96879ef3-708a-4c6b-8fb4-4adc0346c7b4 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:37.239399 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96879ef3-708a-4c6b-8fb4-4adc0346c7b4 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:37.239818 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96879ef3-708a-4c6b-8fb4-4adc0346c7b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:37.275819 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-96879ef3-708a-4c6b-8fb4-4adc0346c7b4 None None] MFA Rules not processed for user `fcfcd9f134e74b09bfdac337c81c8a07`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:37.276724 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-96879ef3-708a-4c6b-8fb4-4adc0346c7b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:37.376437 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1143/2267] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Jun 21 18:29:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:29:37.436500 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96879ef3-708a-4c6b-8fb4-4adc0346c7b4 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:37.436500 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-96879ef3-708a-4c6b-8fb4-4adc0346c7b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:37.456022 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-f00d5114-6eb1-44a9-a8e4-8bc280a1e857 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e33ee5e7c3a84032a521f7981148fbf1'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:37.476045 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-f00d5114-6eb1-44a9-a8e4-8bc280a1e857 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e33ee5e7c3a84032a521f7981148fbf1'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:37.476907 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-f00d5114-6eb1-44a9-a8e4-8bc280a1e857 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e33ee5e7c3a84032a521f7981148fbf1'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:37.487575 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-f00d5114-6eb1-44a9-a8e4-8bc280a1e857 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e33ee5e7c3a84032a521f7981148fbf1'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:37.488618 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-f00d5114-6eb1-44a9-a8e4-8bc280a1e857 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e33ee5e7c3a84032a521f7981148fbf1'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:37.580273 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96879ef3-708a-4c6b-8fb4-4adc0346c7b4 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:37.582426 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1125/2268] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:29:37 2022] DELETE /identity/v3/users/e33ee5e7c3a84032a521f7981148fbf1 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:37.590032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a0b6f5-a147-4301-913a-9d696da8e057 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:37.590517 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a0b6f5-a147-4301-913a-9d696da8e057 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:37.591001 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a0b6f5-a147-4301-913a-9d696da8e057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:37.625595 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-48a0b6f5-a147-4301-913a-9d696da8e057 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:37.627891 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-48a0b6f5-a147-4301-913a-9d696da8e057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:37.724030 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427a85d3-b026-4d5f-b633-892b4ff6a863 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e0d25565f1543d9a7577288fc09119f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:37.730973 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a85d3-b026-4d5f-b633-892b4ff6a863 service swift] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:37.734270 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a85d3-b026-4d5f-b633-892b4ff6a863 service swift] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:37.734818 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a85d3-b026-4d5f-b633-892b4ff6a863 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:37.750619 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-427a85d3-b026-4d5f-b633-892b4ff6a863 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:37.792117 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1126/2269] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:29:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:37.810777 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a0b6f5-a147-4301-913a-9d696da8e057 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:37.812639 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:37.813523 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-48a0b6f5-a147-4301-913a-9d696da8e057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:37.926759 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:37.952312 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a0b6f5-a147-4301-913a-9d696da8e057 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:37.957150 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d59b345-e0cb-43c3-9844-7bc46d2f56a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:37.958815 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d59b345-e0cb-43c3-9844-7bc46d2f56a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:37.959169 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d59b345-e0cb-43c3-9844-7bc46d2f56a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:37.959473 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d59b345-e0cb-43c3-9844-7bc46d2f56a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:37.966381 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427a85d3-b026-4d5f-b633-892b4ff6a863 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:37.969971 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427a85d3-b026-4d5f-b633-892b4ff6a863 service swift] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:37.974302 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d59b345-e0cb-43c3-9844-7bc46d2f56a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:37.976674 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d59b345-e0cb-43c3-9844-7bc46d2f56a5 service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:37.997813 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1144/2270] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Jun 21 18:29:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 567 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 21 18:29:38.017652 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1127/2271] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:29:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:29:38.910245 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427a85d3-b026-4d5f-b633-892b4ff6a863 service swift] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:38.910737 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-427a85d3-b026-4d5f-b633-892b4ff6a863 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:38.968090 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427a85d3-b026-4d5f-b633-892b4ff6a863 service swift] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:38.978237 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8957115-44da-4123-9f59-b641d23b978c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:38.981177 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8957115-44da-4123-9f59-b641d23b978c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:38.982015 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8957115-44da-4123-9f59-b641d23b978c admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:38.983116 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8957115-44da-4123-9f59-b641d23b978c admin admin] PATH_INFO: `/v3/projects/c2efa42f13bd40e58fc33c2f17155631` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:38.986810 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8957115-44da-4123-9f59-b641d23b978c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2efa42f13bd40e58fc33c2f17155631)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:38.997542 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8957115-44da-4123-9f59-b641d23b978c admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:39.268682 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-a8957115-44da-4123-9f59-b641d23b978c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2efa42f13bd40e58fc33c2f17155631', 'request_id': 'req-a8957115-44da-4123-9f59-b641d23b978c'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:39.291921 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-a8957115-44da-4123-9f59-b641d23b978c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2efa42f13bd40e58fc33c2f17155631', 'request_id': 'req-a8957115-44da-4123-9f59-b641d23b978c'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:39.302934 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-a8957115-44da-4123-9f59-b641d23b978c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2efa42f13bd40e58fc33c2f17155631', 'request_id': 'req-a8957115-44da-4123-9f59-b641d23b978c'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:39.439805 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-a8957115-44da-4123-9f59-b641d23b978c admin admin] The token cache is being invalidate because project c2efa42f13bd40e58fc33c2f17155631 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:29:39.439960 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-a8957115-44da-4123-9f59-b641d23b978c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:39.441378 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-a8957115-44da-4123-9f59-b641d23b978c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:29:39.448302 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1145/2272] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:29:38 2022] DELETE /identity/v3/projects/c2efa42f13bd40e58fc33c2f17155631 => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:29:39.449977 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a424c57-1ca2-47ce-9888-5c67d1d76e53 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:39.450820 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a424c57-1ca2-47ce-9888-5c67d1d76e53 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:39.451261 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a424c57-1ca2-47ce-9888-5c67d1d76e53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:39.488896 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-9a424c57-1ca2-47ce-9888-5c67d1d76e53 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:39.491399 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-9a424c57-1ca2-47ce-9888-5c67d1d76e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:39.612615 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1128/2273] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:29:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:39.620646 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729c067-34c7-4017-baae-ad237416c323 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:39.621035 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729c067-34c7-4017-baae-ad237416c323 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:39.621393 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729c067-34c7-4017-baae-ad237416c323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:39.654529 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-2729c067-34c7-4017-baae-ad237416c323 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:39.658116 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-2729c067-34c7-4017-baae-ad237416c323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:39.686591 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1146/2274] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:29:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:29:39.687949 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a424c57-1ca2-47ce-9888-5c67d1d76e53 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:39.707316 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a424c57-1ca2-47ce-9888-5c67d1d76e53 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:39.717662 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eea9ba-374c-46de-9e51-22706119f841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:39.722956 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eea9ba-374c-46de-9e51-22706119f841 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:39.722956 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eea9ba-374c-46de-9e51-22706119f841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:39.722956 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eea9ba-374c-46de-9e51-22706119f841 admin admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:39.722956 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eea9ba-374c-46de-9e51-22706119f841 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:29:39.722956 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eea9ba-374c-46de-9e51-22706119f841 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:39.725365 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eea9ba-374c-46de-9e51-22706119f841 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:39.756369 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1129/2275] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:29:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:29:39.762323 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2729c067-34c7-4017-baae-ad237416c323 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:39.779723 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2729c067-34c7-4017-baae-ad237416c323 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:39.788082 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60fcd6d-e970-4153-b970-dfe6c0c0141c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:39.789749 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60fcd6d-e970-4153-b970-dfe6c0c0141c admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:39.791484 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60fcd6d-e970-4153-b970-dfe6c0c0141c admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:39.792083 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60fcd6d-e970-4153-b970-dfe6c0c0141c admin admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:39.793585 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60fcd6d-e970-4153-b970-dfe6c0c0141c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:39.797557 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60fcd6d-e970-4153-b970-dfe6c0c0141c admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:39.930062 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1147/2276] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:29:39 2022] POST /identity/v3/projects => generated 373 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:29:39.936611 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eea9ba-374c-46de-9e51-22706119f841 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:39.968506 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eea9ba-374c-46de-9e51-22706119f841 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:40.194294 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6936cbc-35e1-4efc-95eb-759bc29e4331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:40.196076 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6936cbc-35e1-4efc-95eb-759bc29e4331 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:40.200355 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6936cbc-35e1-4efc-95eb-759bc29e4331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:40.201125 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6936cbc-35e1-4efc-95eb-759bc29e4331 admin admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:40.202916 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6936cbc-35e1-4efc-95eb-759bc29e4331 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:40.206960 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6936cbc-35e1-4efc-95eb-759bc29e4331 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:40.625009 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1130/2277] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:29:39 2022] POST /identity/v3/users => generated 287 bytes in 689 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:29:40.630551 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60fcd6d-e970-4153-b970-dfe6c0c0141c admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:40.647099 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60fcd6d-e970-4153-b970-dfe6c0c0141c admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:40.654414 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61eb5ba-cdef-4305-a7aa-aebb09321060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:40.654414 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61eb5ba-cdef-4305-a7aa-aebb09321060 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:40.654960 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61eb5ba-cdef-4305-a7aa-aebb09321060 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:40.655466 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61eb5ba-cdef-4305-a7aa-aebb09321060 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:40.656375 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61eb5ba-cdef-4305-a7aa-aebb09321060 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:40.663337 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61eb5ba-cdef-4305-a7aa-aebb09321060 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:40.674181 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1148/2278] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:40 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:29:40.680184 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6936cbc-35e1-4efc-95eb-759bc29e4331 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:40.703821 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6936cbc-35e1-4efc-95eb-759bc29e4331 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:40.714003 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68206ed0-fdd8-40ad-87fe-7a2723ae5afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:40.715457 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68206ed0-fdd8-40ad-87fe-7a2723ae5afc admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:40.716592 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68206ed0-fdd8-40ad-87fe-7a2723ae5afc admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:40.717320 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68206ed0-fdd8-40ad-87fe-7a2723ae5afc admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:40.719826 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68206ed0-fdd8-40ad-87fe-7a2723ae5afc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:40.723408 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68206ed0-fdd8-40ad-87fe-7a2723ae5afc admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:40.742335 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1131/2279] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:29:40 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:29:40.753204 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61eb5ba-cdef-4305-a7aa-aebb09321060 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:40.770533 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61eb5ba-cdef-4305-a7aa-aebb09321060 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:40.778322 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d15ff2-bc08-4923-8f37-9c61b9f40952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:40.780225 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d15ff2-bc08-4923-8f37-9c61b9f40952 admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:40.781019 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d15ff2-bc08-4923-8f37-9c61b9f40952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:40.782567 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d15ff2-bc08-4923-8f37-9c61b9f40952 admin admin] PATH_INFO: `/v3/projects/37f8f17bcdb34325af69ce4ac0bcb709/users/d5939c90f7c64f408a4509b69628b680/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:40.817397 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d15ff2-bc08-4923-8f37-9c61b9f40952 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37f8f17bcdb34325af69ce4ac0bcb709, user_id=d5939c90f7c64f408a4509b69628b680, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:40.821579 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d15ff2-bc08-4923-8f37-9c61b9f40952 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:40.924790 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1149/2280] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:29:40 2022] PUT /identity/v3/projects/37f8f17bcdb34325af69ce4ac0bcb709/users/d5939c90f7c64f408a4509b69628b680/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:29:40.942888 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a103cdb6-1fc0-4358-8ecd-58ee856bcfd5 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:40.943350 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a103cdb6-1fc0-4358-8ecd-58ee856bcfd5 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:40.943822 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a103cdb6-1fc0-4358-8ecd-58ee856bcfd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:41.003602 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-a103cdb6-1fc0-4358-8ecd-58ee856bcfd5 None None] MFA Rules not processed for user `d5939c90f7c64f408a4509b69628b680`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:41.005869 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-a103cdb6-1fc0-4358-8ecd-58ee856bcfd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:41.087042 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:41.143521 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1132/2281] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:29:40 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:29:41.153275 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d15ff2-bc08-4923-8f37-9c61b9f40952 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:41.155051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-b2d15ff2-bc08-4923-8f37-9c61b9f40952 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:41.214173 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:41.279939 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d15ff2-bc08-4923-8f37-9c61b9f40952 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:41.284766 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a123f4-1806-4220-9ce2-67bad6ea5158 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:41.285496 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a123f4-1806-4220-9ce2-67bad6ea5158 service cinder] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:41.285817 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a123f4-1806-4220-9ce2-67bad6ea5158 service cinder] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:41.286093 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a123f4-1806-4220-9ce2-67bad6ea5158 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:41.300029 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a123f4-1806-4220-9ce2-67bad6ea5158 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:41.302110 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a123f4-1806-4220-9ce2-67bad6ea5158 service cinder] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:41.324564 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1150/2282] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:29:41 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:29:42.438286 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb2734-6498-4b9b-b64a-8628af6cb1f7 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:42.439065 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb2734-6498-4b9b-b64a-8628af6cb1f7 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:42.439282 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb2734-6498-4b9b-b64a-8628af6cb1f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:42.487668 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-16cb2734-6498-4b9b-b64a-8628af6cb1f7 None None] MFA Rules not processed for user `59f011e91b684e1288dad5f0ac733c0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:42.491050 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-16cb2734-6498-4b9b-b64a-8628af6cb1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:42.717508 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1133/2283] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:29:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:44.041173 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e12c5-f4c1-47ad-9651-a413cdeae551 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:44.043880 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e12c5-f4c1-47ad-9651-a413cdeae551 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:44.044842 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e12c5-f4c1-47ad-9651-a413cdeae551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:44.089207 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-581e12c5-f4c1-47ad-9651-a413cdeae551 None None] MFA Rules not processed for user `5c21b9f596da493fac827d3573833ed1`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:44.092990 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-581e12c5-f4c1-47ad-9651-a413cdeae551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:44.226879 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1151/2284] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:29:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:48.888428 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc138f8-bac9-44e8-83b1-f859ac2e3740 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:48.888428 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc138f8-bac9-44e8-83b1-f859ac2e3740 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:48.888428 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc138f8-bac9-44e8-83b1-f859ac2e3740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:48.916166 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-0fc138f8-bac9-44e8-83b1-f859ac2e3740 None None] MFA Rules not processed for user `fcfcd9f134e74b09bfdac337c81c8a07`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:29:48.918537 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-0fc138f8-bac9-44e8-83b1-f859ac2e3740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:49.052576 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1134/2285] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Jun 21 18:29:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:29:49.070474 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581e12c5-f4c1-47ad-9651-a413cdeae551 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:29:49.080758 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-581e12c5-f4c1-47ad-9651-a413cdeae551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:29:49.232936 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581e12c5-f4c1-47ad-9651-a413cdeae551 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:29:49.266915 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae5461c-5031-4df5-956f-f54f811d6433 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e0d25565f1543d9a7577288fc09119f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:29:49.268443 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae5461c-5031-4df5-956f-f54f811d6433 service swift] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:29:49.268817 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae5461c-5031-4df5-956f-f54f811d6433 service swift] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:29:49.269390 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae5461c-5031-4df5-956f-f54f811d6433 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:29:49.283910 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae5461c-5031-4df5-956f-f54f811d6433 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:29:49.285813 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae5461c-5031-4df5-956f-f54f811d6433 service swift] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:29:49.299884 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1152/2286] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Jun 21 18:29:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 18:29:59.043296 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=62 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:29:59.293744 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=77 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:02.058280 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc138f8-bac9-44e8-83b1-f859ac2e3740 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:02.065171 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-0fc138f8-bac9-44e8-83b1-f859ac2e3740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:02.300019 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc138f8-bac9-44e8-83b1-f859ac2e3740 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:02.310973 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb5d9b4-08bb-47ac-92d5-e5fd7ed1a71a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:02.315701 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb5d9b4-08bb-47ac-92d5-e5fd7ed1a71a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:02.316637 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb5d9b4-08bb-47ac-92d5-e5fd7ed1a71a admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:02.317040 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb5d9b4-08bb-47ac-92d5-e5fd7ed1a71a admin admin] PATH_INFO: `/v3/users/b8521e4b29434f04b51b9bcabab0e5b1` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:02.320858 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb5d9b4-08bb-47ac-92d5-e5fd7ed1a71a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8521e4b29434f04b51b9bcabab0e5b1)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:02.327489 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb5d9b4-08bb-47ac-92d5-e5fd7ed1a71a admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:02.766267 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-1fb5d9b4-08bb-47ac-92d5-e5fd7ed1a71a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8521e4b29434f04b51b9bcabab0e5b1'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:02.856568 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-1fb5d9b4-08bb-47ac-92d5-e5fd7ed1a71a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8521e4b29434f04b51b9bcabab0e5b1'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:02.860951 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-1fb5d9b4-08bb-47ac-92d5-e5fd7ed1a71a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8521e4b29434f04b51b9bcabab0e5b1'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:02.905850 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-1fb5d9b4-08bb-47ac-92d5-e5fd7ed1a71a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8521e4b29434f04b51b9bcabab0e5b1'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:02.907501 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-1fb5d9b4-08bb-47ac-92d5-e5fd7ed1a71a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8521e4b29434f04b51b9bcabab0e5b1'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:03.206584 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1135/2287] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:30:02 2022] DELETE /identity/v3/users/b8521e4b29434f04b51b9bcabab0e5b1 => generated 0 bytes in 1152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:03.214838 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b5c4f2-4805-40fe-9aee-b3d227d85aef None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:03.215549 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b5c4f2-4805-40fe-9aee-b3d227d85aef None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:03.215549 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b5c4f2-4805-40fe-9aee-b3d227d85aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:03.255802 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-89b5c4f2-4805-40fe-9aee-b3d227d85aef None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:03.255802 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-89b5c4f2-4805-40fe-9aee-b3d227d85aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:03.465866 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1153/2288] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:30:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:30:03.497267 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb5d9b4-08bb-47ac-92d5-e5fd7ed1a71a admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:03.498898 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-1fb5d9b4-08bb-47ac-92d5-e5fd7ed1a71a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:03.723586 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb5d9b4-08bb-47ac-92d5-e5fd7ed1a71a admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:03.733055 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01383f71-5053-4027-b1c6-bae8fcd58765 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:03.733785 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01383f71-5053-4027-b1c6-bae8fcd58765 service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:03.734054 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01383f71-5053-4027-b1c6-bae8fcd58765 service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:03.734337 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01383f71-5053-4027-b1c6-bae8fcd58765 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:03.752549 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01383f71-5053-4027-b1c6-bae8fcd58765 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:03.754957 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01383f71-5053-4027-b1c6-bae8fcd58765 service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:03.779706 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1136/2289] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:30:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:30:04.178355 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b5c4f2-4805-40fe-9aee-b3d227d85aef None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:04.185875 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-89b5c4f2-4805-40fe-9aee-b3d227d85aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:04.517297 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b5c4f2-4805-40fe-9aee-b3d227d85aef None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:04.544031 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb87266a-d0b3-400d-bd5d-45d93c205748 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938187967efe49be9c5389d1feef81ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:04.545452 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb87266a-d0b3-400d-bd5d-45d93c205748 service placement] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:04.545743 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb87266a-d0b3-400d-bd5d-45d93c205748 service placement] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:04.546143 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb87266a-d0b3-400d-bd5d-45d93c205748 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:04.548796 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-cb87266a-d0b3-400d-bd5d-45d93c205748 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:04.573187 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb87266a-d0b3-400d-bd5d-45d93c205748 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:04.575663 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb87266a-d0b3-400d-bd5d-45d93c205748 service placement] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:04.627724 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1154/2290] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:30:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:30:05.169621 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01383f71-5053-4027-b1c6-bae8fcd58765 service neutron] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:05.185120 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-01383f71-5053-4027-b1c6-bae8fcd58765 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:05.261652 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01383f71-5053-4027-b1c6-bae8fcd58765 service neutron] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:05.278031 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08210bef-22f3-4392-ad8f-3174a4ab40db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:05.299487 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08210bef-22f3-4392-ad8f-3174a4ab40db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:05.301123 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08210bef-22f3-4392-ad8f-3174a4ab40db admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:05.302040 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08210bef-22f3-4392-ad8f-3174a4ab40db admin admin] PATH_INFO: `/v3/projects/b9b64f5e855745b4800958294d93315e` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:05.308186 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08210bef-22f3-4392-ad8f-3174a4ab40db admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9b64f5e855745b4800958294d93315e)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:05.316430 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08210bef-22f3-4392-ad8f-3174a4ab40db admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:05.365187 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:05.576193 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-08210bef-22f3-4392-ad8f-3174a4ab40db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9b64f5e855745b4800958294d93315e', 'request_id': 'req-08210bef-22f3-4392-ad8f-3174a4ab40db'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:05.741811 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-08210bef-22f3-4392-ad8f-3174a4ab40db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9b64f5e855745b4800958294d93315e', 'request_id': 'req-08210bef-22f3-4392-ad8f-3174a4ab40db'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:05.767118 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-08210bef-22f3-4392-ad8f-3174a4ab40db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9b64f5e855745b4800958294d93315e', 'request_id': 'req-08210bef-22f3-4392-ad8f-3174a4ab40db'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:05.797870 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-08210bef-22f3-4392-ad8f-3174a4ab40db admin admin] The token cache is being invalidate because project b9b64f5e855745b4800958294d93315e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:30:05.798600 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-08210bef-22f3-4392-ad8f-3174a4ab40db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:05.799692 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-08210bef-22f3-4392-ad8f-3174a4ab40db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:05.806049 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1137/2291] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:30:05 2022] DELETE /identity/v3/projects/b9b64f5e855745b4800958294d93315e => generated 0 bytes in 640 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:05.814128 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eba4a5-ce93-45fe-a164-cb2f26abfda6 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:05.814851 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eba4a5-ce93-45fe-a164-cb2f26abfda6 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:05.816838 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eba4a5-ce93-45fe-a164-cb2f26abfda6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:05.873228 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-56eba4a5-ce93-45fe-a164-cb2f26abfda6 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:05.874904 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-56eba4a5-ce93-45fe-a164-cb2f26abfda6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:06.025890 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1155/2292] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:30:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:06.032550 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f30f17-c30d-4606-bba0-e2a0cc6e7e51 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:06.032840 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f30f17-c30d-4606-bba0-e2a0cc6e7e51 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:06.033299 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f30f17-c30d-4606-bba0-e2a0cc6e7e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:06.099362 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-d1f30f17-c30d-4606-bba0-e2a0cc6e7e51 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:06.106383 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-d1f30f17-c30d-4606-bba0-e2a0cc6e7e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:06.152506 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1138/2293] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:30:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:06.153812 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eba4a5-ce93-45fe-a164-cb2f26abfda6 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:06.181109 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eba4a5-ce93-45fe-a164-cb2f26abfda6 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:06.197118 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9484855-0c2e-402f-96d7-36326f9469f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:06.198001 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9484855-0c2e-402f-96d7-36326f9469f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:06.198302 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9484855-0c2e-402f-96d7-36326f9469f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:06.198596 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9484855-0c2e-402f-96d7-36326f9469f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:06.199988 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9484855-0c2e-402f-96d7-36326f9469f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:30:06.199988 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9484855-0c2e-402f-96d7-36326f9469f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:06.204263 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9484855-0c2e-402f-96d7-36326f9469f5 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:06.237159 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:06.280229 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1156/2294] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:30:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:30:06.288112 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f30f17-c30d-4606-bba0-e2a0cc6e7e51 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:06.311994 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f30f17-c30d-4606-bba0-e2a0cc6e7e51 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:06.329685 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a41d21-df45-45ad-a78a-dcd1de835f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:06.330768 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a41d21-df45-45ad-a78a-dcd1de835f7a admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:06.331135 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a41d21-df45-45ad-a78a-dcd1de835f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:06.333757 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a41d21-df45-45ad-a78a-dcd1de835f7a admin admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:06.333757 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a41d21-df45-45ad-a78a-dcd1de835f7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:06.335330 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a41d21-df45-45ad-a78a-dcd1de835f7a admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:06.490551 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1139/2295] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:30:06 2022] POST /identity/v3/projects => generated 397 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:30:06.497594 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9484855-0c2e-402f-96d7-36326f9469f5 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:06.515251 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9484855-0c2e-402f-96d7-36326f9469f5 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:06.649076 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ffeec3-4de1-4981-a88e-b1bb2729a852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:06.651084 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ffeec3-4de1-4981-a88e-b1bb2729a852 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:06.651384 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ffeec3-4de1-4981-a88e-b1bb2729a852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:06.651654 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ffeec3-4de1-4981-a88e-b1bb2729a852 admin admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:06.652421 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ffeec3-4de1-4981-a88e-b1bb2729a852 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:06.657537 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ffeec3-4de1-4981-a88e-b1bb2729a852 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:06.793538 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a41d21-df45-45ad-a78a-dcd1de835f7a admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:06.795272 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1157/2296] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:30:06 2022] POST /identity/v3/users => generated 299 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:30:06.818479 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a41d21-df45-45ad-a78a-dcd1de835f7a admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:06.831950 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e623754-0576-426e-bce1-b272231662bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:06.833461 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e623754-0576-426e-bce1-b272231662bb admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:06.834443 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e623754-0576-426e-bce1-b272231662bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:06.835003 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e623754-0576-426e-bce1-b272231662bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:06.839430 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e623754-0576-426e-bce1-b272231662bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:06.843795 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e623754-0576-426e-bce1-b272231662bb admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:06.867719 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ffeec3-4de1-4981-a88e-b1bb2729a852 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:06.884325 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1140/2297] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:30:06.894810 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ffeec3-4de1-4981-a88e-b1bb2729a852 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:06.913021 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387fcd4d-fbb7-46a6-8d0c-be55b5118478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:06.914220 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387fcd4d-fbb7-46a6-8d0c-be55b5118478 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:06.918249 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387fcd4d-fbb7-46a6-8d0c-be55b5118478 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:06.919652 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387fcd4d-fbb7-46a6-8d0c-be55b5118478 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:06.922114 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387fcd4d-fbb7-46a6-8d0c-be55b5118478 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:06.924800 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387fcd4d-fbb7-46a6-8d0c-be55b5118478 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:06.947548 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e623754-0576-426e-bce1-b272231662bb admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:06.956958 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1158/2298] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:30:06.974598 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e623754-0576-426e-bce1-b272231662bb admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:06.983170 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5885781c-4904-4c53-b9c2-42f54e67377e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:06.984490 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5885781c-4904-4c53-b9c2-42f54e67377e admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:06.984956 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5885781c-4904-4c53-b9c2-42f54e67377e admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:06.985402 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5885781c-4904-4c53-b9c2-42f54e67377e admin admin] PATH_INFO: `/v3/projects/cfa7b247fe3b48419a2810ab33a0556b/users/a946dc4f81db4ef9a1a8f17d38115e3c/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:07.048185 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5885781c-4904-4c53-b9c2-42f54e67377e admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfa7b247fe3b48419a2810ab33a0556b, user_id=a946dc4f81db4ef9a1a8f17d38115e3c, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:07.053489 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5885781c-4904-4c53-b9c2-42f54e67377e admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:07.206819 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42d864e-b09b-4f00-80af-1602c0c35cf6 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:07.207275 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42d864e-b09b-4f00-80af-1602c0c35cf6 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:07.207668 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42d864e-b09b-4f00-80af-1602c0c35cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:07.210244 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1141/2299] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:30:06 2022] PUT /identity/v3/projects/cfa7b247fe3b48419a2810ab33a0556b/users/a946dc4f81db4ef9a1a8f17d38115e3c/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:07.261509 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-e42d864e-b09b-4f00-80af-1602c0c35cf6 None None] MFA Rules not processed for user `a946dc4f81db4ef9a1a8f17d38115e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:07.262773 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-e42d864e-b09b-4f00-80af-1602c0c35cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:07.387506 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1159/2300] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:30:07 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:07.415849 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5885781c-4904-4c53-b9c2-42f54e67377e admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:07.417901 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-5885781c-4904-4c53-b9c2-42f54e67377e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:07.708680 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5885781c-4904-4c53-b9c2-42f54e67377e admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:07.726708 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491c56ce-4101-454a-9c98-08fa8d75fd80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:07.728822 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c56ce-4101-454a-9c98-08fa8d75fd80 service cinder] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:07.741222 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c56ce-4101-454a-9c98-08fa8d75fd80 service cinder] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:07.749171 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c56ce-4101-454a-9c98-08fa8d75fd80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:07.796371 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491c56ce-4101-454a-9c98-08fa8d75fd80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:07.807560 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491c56ce-4101-454a-9c98-08fa8d75fd80 service cinder] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:07.873349 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1142/2301] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:30:07 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:30:07.906859 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42d864e-b09b-4f00-80af-1602c0c35cf6 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:07.906859 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-e42d864e-b09b-4f00-80af-1602c0c35cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:08.197454 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42d864e-b09b-4f00-80af-1602c0c35cf6 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:08.220878 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03076ebc-7bb6-4408-bb44-4009b08569ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:08.220878 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03076ebc-7bb6-4408-bb44-4009b08569ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:08.220878 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03076ebc-7bb6-4408-bb44-4009b08569ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:08.220878 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03076ebc-7bb6-4408-bb44-4009b08569ba admin admin] PATH_INFO: `/v3/users/d5939c90f7c64f408a4509b69628b680` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:08.221588 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03076ebc-7bb6-4408-bb44-4009b08569ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5939c90f7c64f408a4509b69628b680)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:08.226605 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03076ebc-7bb6-4408-bb44-4009b08569ba admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:08.248561 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:08.795671 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-03076ebc-7bb6-4408-bb44-4009b08569ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5939c90f7c64f408a4509b69628b680'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:08.822925 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-03076ebc-7bb6-4408-bb44-4009b08569ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5939c90f7c64f408a4509b69628b680'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:08.952310 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-03076ebc-7bb6-4408-bb44-4009b08569ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5939c90f7c64f408a4509b69628b680'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:08.953779 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-03076ebc-7bb6-4408-bb44-4009b08569ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5939c90f7c64f408a4509b69628b680'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:08.964043 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-03076ebc-7bb6-4408-bb44-4009b08569ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5939c90f7c64f408a4509b69628b680'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:08.999917 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1160/2302] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:30:07 2022] DELETE /identity/v3/users/d5939c90f7c64f408a4509b69628b680 => generated 0 bytes in 1098 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:09.003183 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f706d607-2a1e-4cde-b193-4b6a97784403 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:09.005104 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f706d607-2a1e-4cde-b193-4b6a97784403 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:09.006055 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f706d607-2a1e-4cde-b193-4b6a97784403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:09.077447 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-f706d607-2a1e-4cde-b193-4b6a97784403 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:09.080999 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-f706d607-2a1e-4cde-b193-4b6a97784403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:09.278900 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:09.347235 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1143/2303] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:30:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:09.361321 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03076ebc-7bb6-4408-bb44-4009b08569ba admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:09.361321 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-03076ebc-7bb6-4408-bb44-4009b08569ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:09.485591 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03076ebc-7bb6-4408-bb44-4009b08569ba admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:09.495307 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36872b62-8ae2-47e3-b404-f043dd565a49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:09.496294 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36872b62-8ae2-47e3-b404-f043dd565a49 service neutron] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:09.496852 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36872b62-8ae2-47e3-b404-f043dd565a49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:09.497420 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36872b62-8ae2-47e3-b404-f043dd565a49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:09.525684 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36872b62-8ae2-47e3-b404-f043dd565a49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:09.545863 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36872b62-8ae2-47e3-b404-f043dd565a49 service neutron] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:09.547596 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1161/2304] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:30:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:30:10.736506 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f706d607-2a1e-4cde-b193-4b6a97784403 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:10.738076 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-f706d607-2a1e-4cde-b193-4b6a97784403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:10.776764 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f706d607-2a1e-4cde-b193-4b6a97784403 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:10.797337 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90133762-ec19-485d-bdf2-fcd0f5423f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:10.798260 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90133762-ec19-485d-bdf2-fcd0f5423f8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:10.798548 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90133762-ec19-485d-bdf2-fcd0f5423f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:10.798830 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90133762-ec19-485d-bdf2-fcd0f5423f8f admin admin] PATH_INFO: `/v3/projects/37f8f17bcdb34325af69ce4ac0bcb709` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:10.804787 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90133762-ec19-485d-bdf2-fcd0f5423f8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=37f8f17bcdb34325af69ce4ac0bcb709)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:10.806963 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90133762-ec19-485d-bdf2-fcd0f5423f8f admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:11.122424 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-90133762-ec19-485d-bdf2-fcd0f5423f8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37f8f17bcdb34325af69ce4ac0bcb709', 'request_id': 'req-90133762-ec19-485d-bdf2-fcd0f5423f8f'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:11.250984 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-90133762-ec19-485d-bdf2-fcd0f5423f8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37f8f17bcdb34325af69ce4ac0bcb709', 'request_id': 'req-90133762-ec19-485d-bdf2-fcd0f5423f8f'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:11.261588 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-90133762-ec19-485d-bdf2-fcd0f5423f8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37f8f17bcdb34325af69ce4ac0bcb709', 'request_id': 'req-90133762-ec19-485d-bdf2-fcd0f5423f8f'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:11.316272 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-90133762-ec19-485d-bdf2-fcd0f5423f8f admin admin] The token cache is being invalidate because project 37f8f17bcdb34325af69ce4ac0bcb709 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:30:11.316959 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-90133762-ec19-485d-bdf2-fcd0f5423f8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:11.319799 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-90133762-ec19-485d-bdf2-fcd0f5423f8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:11.325221 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1144/2305] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:30:10 2022] DELETE /identity/v3/projects/37f8f17bcdb34325af69ce4ac0bcb709 => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:11.332989 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ee0d2-4b5e-4d75-82ca-190271a9c6d6 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:11.333829 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ee0d2-4b5e-4d75-82ca-190271a9c6d6 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:11.334309 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ee0d2-4b5e-4d75-82ca-190271a9c6d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:11.362571 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-280ee0d2-4b5e-4d75-82ca-190271a9c6d6 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:11.366231 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-280ee0d2-4b5e-4d75-82ca-190271a9c6d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:11.500682 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1162/2306] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:30:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:11.511434 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e602af-8082-479a-8f6d-527c92eb93b5 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:11.512424 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e602af-8082-479a-8f6d-527c92eb93b5 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:11.512954 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e602af-8082-479a-8f6d-527c92eb93b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:11.541185 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-74e602af-8082-479a-8f6d-527c92eb93b5 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:11.542644 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-74e602af-8082-479a-8f6d-527c92eb93b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:11.551203 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1145/2307] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:30:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:11.556071 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280ee0d2-4b5e-4d75-82ca-190271a9c6d6 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:11.574200 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280ee0d2-4b5e-4d75-82ca-190271a9c6d6 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:11.581794 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62929def-ea09-49b4-bea5-e7328273b832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:11.582839 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62929def-ea09-49b4-bea5-e7328273b832 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:11.583286 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62929def-ea09-49b4-bea5-e7328273b832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:11.583731 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62929def-ea09-49b4-bea5-e7328273b832 admin admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:11.585039 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62929def-ea09-49b4-bea5-e7328273b832 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:30:11.585539 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62929def-ea09-49b4-bea5-e7328273b832 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:11.588555 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62929def-ea09-49b4-bea5-e7328273b832 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:11.608233 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1163/2308] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:30:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:30:11.624020 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e602af-8082-479a-8f6d-527c92eb93b5 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:11.640556 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e602af-8082-479a-8f6d-527c92eb93b5 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:11.650275 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b97bec-b769-4158-ad21-2b160ecdf458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:11.651245 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b97bec-b769-4158-ad21-2b160ecdf458 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:11.656292 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b97bec-b769-4158-ad21-2b160ecdf458 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:11.656292 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b97bec-b769-4158-ad21-2b160ecdf458 admin admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:11.656292 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b97bec-b769-4158-ad21-2b160ecdf458 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:11.656292 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b97bec-b769-4158-ad21-2b160ecdf458 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:11.864852 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1146/2309] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:30:11 2022] POST /identity/v3/projects => generated 371 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:30:11.868935 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62929def-ea09-49b4-bea5-e7328273b832 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:11.910630 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62929def-ea09-49b4-bea5-e7328273b832 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:11.937128 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:11.992436 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc44a28c-fa67-4442-8892-eabb4d2ee8c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:11.994045 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44a28c-fa67-4442-8892-eabb4d2ee8c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:11.994545 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44a28c-fa67-4442-8892-eabb4d2ee8c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:11.995030 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44a28c-fa67-4442-8892-eabb4d2ee8c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:11.996266 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc44a28c-fa67-4442-8892-eabb4d2ee8c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:11.999164 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc44a28c-fa67-4442-8892-eabb4d2ee8c3 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:12.106269 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1164/2310] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:30:11 2022] POST /identity/v3/users => generated 286 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:30:12.110417 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b97bec-b769-4158-ad21-2b160ecdf458 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:12.121715 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b97bec-b769-4158-ad21-2b160ecdf458 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:12.127324 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e5c139-5174-40e3-adbf-9644923dd805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:12.128010 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e5c139-5174-40e3-adbf-9644923dd805 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:12.128299 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e5c139-5174-40e3-adbf-9644923dd805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:12.128614 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e5c139-5174-40e3-adbf-9644923dd805 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:12.129403 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e5c139-5174-40e3-adbf-9644923dd805 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:12.131131 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e5c139-5174-40e3-adbf-9644923dd805 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:12.147890 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1147/2311] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:30:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:30:12.153069 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc44a28c-fa67-4442-8892-eabb4d2ee8c3 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:12.164610 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc44a28c-fa67-4442-8892-eabb4d2ee8c3 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:12.170844 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7718a4ba-1c2f-4492-8038-d1b4ea900129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:12.171842 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7718a4ba-1c2f-4492-8038-d1b4ea900129 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:12.171842 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7718a4ba-1c2f-4492-8038-d1b4ea900129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:12.172230 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7718a4ba-1c2f-4492-8038-d1b4ea900129 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:12.173029 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7718a4ba-1c2f-4492-8038-d1b4ea900129 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:12.174659 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7718a4ba-1c2f-4492-8038-d1b4ea900129 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:12.186817 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1165/2312] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:30:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:30:12.191317 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e5c139-5174-40e3-adbf-9644923dd805 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:12.202205 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e5c139-5174-40e3-adbf-9644923dd805 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:12.207970 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc87be1d-804a-4337-b70b-d2ecce39a863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:12.208943 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc87be1d-804a-4337-b70b-d2ecce39a863 admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:12.209243 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc87be1d-804a-4337-b70b-d2ecce39a863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:12.209506 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc87be1d-804a-4337-b70b-d2ecce39a863 admin admin] PATH_INFO: `/v3/projects/65efebfec3f0401190b3116e52d634e8/users/113b17a680c640f19b0f402f371c6cbb/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:12.232697 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc87be1d-804a-4337-b70b-d2ecce39a863 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65efebfec3f0401190b3116e52d634e8, user_id=113b17a680c640f19b0f402f371c6cbb, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:12.235599 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc87be1d-804a-4337-b70b-d2ecce39a863 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:12.345778 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1148/2313] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:30:12 2022] PUT /identity/v3/projects/65efebfec3f0401190b3116e52d634e8/users/113b17a680c640f19b0f402f371c6cbb/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:12.351779 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2e9229-2a2a-4ae7-9f31-60c75088d0fe None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:12.353202 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2e9229-2a2a-4ae7-9f31-60c75088d0fe None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:12.353202 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2e9229-2a2a-4ae7-9f31-60c75088d0fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:12.394516 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-3d2e9229-2a2a-4ae7-9f31-60c75088d0fe None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:12.396537 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-3d2e9229-2a2a-4ae7-9f31-60c75088d0fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:12.481167 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1166/2314] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:30:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:12.496503 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc87be1d-804a-4337-b70b-d2ecce39a863 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:12.498408 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-cc87be1d-804a-4337-b70b-d2ecce39a863 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:12.609730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc87be1d-804a-4337-b70b-d2ecce39a863 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:12.616039 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7c558a-0e7c-4932-ab1e-a391ebd0c7f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:12.617146 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c558a-0e7c-4932-ab1e-a391ebd0c7f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:12.617581 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c558a-0e7c-4932-ab1e-a391ebd0c7f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:12.618558 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c558a-0e7c-4932-ab1e-a391ebd0c7f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:12.636052 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7c558a-0e7c-4932-ab1e-a391ebd0c7f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:12.638616 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7c558a-0e7c-4932-ab1e-a391ebd0c7f6 service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:12.657286 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1149/2315] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:30:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:30:15.850491 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd484ca5-800d-4653-aabc-6eda057208aa None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:15.854601 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd484ca5-800d-4653-aabc-6eda057208aa None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:15.854601 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd484ca5-800d-4653-aabc-6eda057208aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:15.940629 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-bd484ca5-800d-4653-aabc-6eda057208aa None None] MFA Rules not processed for user `fcfcd9f134e74b09bfdac337c81c8a07`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:15.952488 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-bd484ca5-800d-4653-aabc-6eda057208aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:16.074341 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1167/2316] 10.222.0.23 () {58 vars in 967 bytes} [Tue Jun 21 18:30:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:16.093867 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7c558a-0e7c-4932-ab1e-a391ebd0c7f6 service neutron] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:16.095813 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-3b7c558a-0e7c-4932-ab1e-a391ebd0c7f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:16.114375 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:16.254458 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7c558a-0e7c-4932-ab1e-a391ebd0c7f6 service neutron] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:16.257547 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b385d-bf69-4b55-a99a-e354009ae666 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e0d25565f1543d9a7577288fc09119f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:16.259200 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b385d-bf69-4b55-a99a-e354009ae666 service swift] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:16.259200 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b385d-bf69-4b55-a99a-e354009ae666 service swift] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:16.259438 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b385d-bf69-4b55-a99a-e354009ae666 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:16.269151 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544b385d-bf69-4b55-a99a-e354009ae666 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:16.271163 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544b385d-bf69-4b55-a99a-e354009ae666 service swift] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:16.282082 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1150/2317] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Jun 21 18:30:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 18:30:19.259941 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd484ca5-800d-4653-aabc-6eda057208aa None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:19.266198 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-bd484ca5-800d-4653-aabc-6eda057208aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:19.277241 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd484ca5-800d-4653-aabc-6eda057208aa None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:19.282927 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39267f3e-836b-425b-8b1f-cd92a7ae3758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:19.287150 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39267f3e-836b-425b-8b1f-cd92a7ae3758 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:19.287439 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39267f3e-836b-425b-8b1f-cd92a7ae3758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:19.287971 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39267f3e-836b-425b-8b1f-cd92a7ae3758 admin admin] PATH_INFO: `/v3/users/a946dc4f81db4ef9a1a8f17d38115e3c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:19.290536 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39267f3e-836b-425b-8b1f-cd92a7ae3758 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a946dc4f81db4ef9a1a8f17d38115e3c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:19.292629 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39267f3e-836b-425b-8b1f-cd92a7ae3758 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:19.294913 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:19.543070 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-39267f3e-836b-425b-8b1f-cd92a7ae3758 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a946dc4f81db4ef9a1a8f17d38115e3c'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:19.579125 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-39267f3e-836b-425b-8b1f-cd92a7ae3758 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a946dc4f81db4ef9a1a8f17d38115e3c'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:19.692988 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-39267f3e-836b-425b-8b1f-cd92a7ae3758 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a946dc4f81db4ef9a1a8f17d38115e3c'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:19.693785 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-39267f3e-836b-425b-8b1f-cd92a7ae3758 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a946dc4f81db4ef9a1a8f17d38115e3c'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:19.705240 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-39267f3e-836b-425b-8b1f-cd92a7ae3758 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a946dc4f81db4ef9a1a8f17d38115e3c'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:19.713149 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1168/2318] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:30:19 2022] DELETE /identity/v3/users/a946dc4f81db4ef9a1a8f17d38115e3c => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:19.718432 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21105c3-9b05-4b25-b9da-bfa754ca7e25 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:19.720078 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21105c3-9b05-4b25-b9da-bfa754ca7e25 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:19.722910 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21105c3-9b05-4b25-b9da-bfa754ca7e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:19.770484 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-e21105c3-9b05-4b25-b9da-bfa754ca7e25 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:19.771852 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-e21105c3-9b05-4b25-b9da-bfa754ca7e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:19.991212 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1151/2319] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:30:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:20.017856 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39267f3e-836b-425b-8b1f-cd92a7ae3758 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:20.022989 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-39267f3e-836b-425b-8b1f-cd92a7ae3758 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:20.157918 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39267f3e-836b-425b-8b1f-cd92a7ae3758 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:20.171639 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17108768-6fc4-4c1d-ae8e-43b89b9f3df9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:20.172784 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17108768-6fc4-4c1d-ae8e-43b89b9f3df9 service neutron] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:20.173154 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17108768-6fc4-4c1d-ae8e-43b89b9f3df9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:20.173665 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17108768-6fc4-4c1d-ae8e-43b89b9f3df9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:20.199861 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17108768-6fc4-4c1d-ae8e-43b89b9f3df9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:20.203338 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17108768-6fc4-4c1d-ae8e-43b89b9f3df9 service neutron] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:20.242513 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1169/2320] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:30:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:30:20.661563 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21105c3-9b05-4b25-b9da-bfa754ca7e25 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:20.663700 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-e21105c3-9b05-4b25-b9da-bfa754ca7e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:20.694342 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21105c3-9b05-4b25-b9da-bfa754ca7e25 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:20.706613 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe1f22e-61d1-4659-bc21-3cde76e1ddb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:20.708194 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1f22e-61d1-4659-bc21-3cde76e1ddb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:20.709004 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1f22e-61d1-4659-bc21-3cde76e1ddb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:20.709518 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1f22e-61d1-4659-bc21-3cde76e1ddb3 admin admin] PATH_INFO: `/v3/users/b6bbd45ee6964c4986f3c778ab8ad231` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:20.723471 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe1f22e-61d1-4659-bc21-3cde76e1ddb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6bbd45ee6964c4986f3c778ab8ad231)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:20.727205 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe1f22e-61d1-4659-bc21-3cde76e1ddb3 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:21.041511 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-efe1f22e-61d1-4659-bc21-3cde76e1ddb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6bbd45ee6964c4986f3c778ab8ad231'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:21.073108 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-efe1f22e-61d1-4659-bc21-3cde76e1ddb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6bbd45ee6964c4986f3c778ab8ad231'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:21.078257 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-efe1f22e-61d1-4659-bc21-3cde76e1ddb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6bbd45ee6964c4986f3c778ab8ad231'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:21.098301 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:21.102860 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-efe1f22e-61d1-4659-bc21-3cde76e1ddb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6bbd45ee6964c4986f3c778ab8ad231'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:21.104802 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-efe1f22e-61d1-4659-bc21-3cde76e1ddb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6bbd45ee6964c4986f3c778ab8ad231'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:21.230442 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1152/2321] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:30:20 2022] DELETE /identity/v3/users/b6bbd45ee6964c4986f3c778ab8ad231 => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:21.783416 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17108768-6fc4-4c1d-ae8e-43b89b9f3df9 service neutron] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:21.800756 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17108768-6fc4-4c1d-ae8e-43b89b9f3df9 service neutron] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:21.905585 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459454ea-507c-4c3f-8102-49f64ad84cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:21.920946 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459454ea-507c-4c3f-8102-49f64ad84cf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:21.922520 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459454ea-507c-4c3f-8102-49f64ad84cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:21.923297 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459454ea-507c-4c3f-8102-49f64ad84cf0 admin admin] PATH_INFO: `/v3/projects/444a0fac8c834c3cab5ba1df5a261aac` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:21.931365 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459454ea-507c-4c3f-8102-49f64ad84cf0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=444a0fac8c834c3cab5ba1df5a261aac)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:21.935153 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459454ea-507c-4c3f-8102-49f64ad84cf0 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:21.982961 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe1f22e-61d1-4659-bc21-3cde76e1ddb3 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:21.984500 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-efe1f22e-61d1-4659-bc21-3cde76e1ddb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:22.005722 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe1f22e-61d1-4659-bc21-3cde76e1ddb3 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:22.017742 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:22.020864 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3278d4f-c396-437f-8453-89b8d22c7fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:22.022015 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3278d4f-c396-437f-8453-89b8d22c7fbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:22.024729 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3278d4f-c396-437f-8453-89b8d22c7fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:22.024729 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3278d4f-c396-437f-8453-89b8d22c7fbf admin admin] PATH_INFO: `/v3/projects/cfa7b247fe3b48419a2810ab33a0556b` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:22.025180 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3278d4f-c396-437f-8453-89b8d22c7fbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfa7b247fe3b48419a2810ab33a0556b)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:22.027345 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3278d4f-c396-437f-8453-89b8d22c7fbf admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:22.161316 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-459454ea-507c-4c3f-8102-49f64ad84cf0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '444a0fac8c834c3cab5ba1df5a261aac', 'request_id': 'req-459454ea-507c-4c3f-8102-49f64ad84cf0'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:22.174101 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-459454ea-507c-4c3f-8102-49f64ad84cf0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '444a0fac8c834c3cab5ba1df5a261aac', 'request_id': 'req-459454ea-507c-4c3f-8102-49f64ad84cf0'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:22.183093 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-459454ea-507c-4c3f-8102-49f64ad84cf0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '444a0fac8c834c3cab5ba1df5a261aac', 'request_id': 'req-459454ea-507c-4c3f-8102-49f64ad84cf0'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:22.199679 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-e3278d4f-c396-437f-8453-89b8d22c7fbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfa7b247fe3b48419a2810ab33a0556b', 'request_id': 'req-e3278d4f-c396-437f-8453-89b8d22c7fbf'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:22.301669 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-459454ea-507c-4c3f-8102-49f64ad84cf0 admin admin] The token cache is being invalidate because project 444a0fac8c834c3cab5ba1df5a261aac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:30:22.302087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-459454ea-507c-4c3f-8102-49f64ad84cf0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:22.302854 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-459454ea-507c-4c3f-8102-49f64ad84cf0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:22.305281 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1170/2322] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:30:21 2022] DELETE /identity/v3/projects/444a0fac8c834c3cab5ba1df5a261aac => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:30:22.313989 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396bcff8-f5e9-4417-bb7d-b7431337386d None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:22.314487 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396bcff8-f5e9-4417-bb7d-b7431337386d None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:22.315000 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396bcff8-f5e9-4417-bb7d-b7431337386d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:22.366239 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-396bcff8-f5e9-4417-bb7d-b7431337386d None None] MFA Rules not processed for user `113b17a680c640f19b0f402f371c6cbb`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:22.367676 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-396bcff8-f5e9-4417-bb7d-b7431337386d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:22.377583 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-e3278d4f-c396-437f-8453-89b8d22c7fbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfa7b247fe3b48419a2810ab33a0556b', 'request_id': 'req-e3278d4f-c396-437f-8453-89b8d22c7fbf'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:22.388929 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-e3278d4f-c396-437f-8453-89b8d22c7fbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfa7b247fe3b48419a2810ab33a0556b', 'request_id': 'req-e3278d4f-c396-437f-8453-89b8d22c7fbf'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:22.421426 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-e3278d4f-c396-437f-8453-89b8d22c7fbf admin admin] The token cache is being invalidate because project cfa7b247fe3b48419a2810ab33a0556b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:30:22.423640 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-e3278d4f-c396-437f-8453-89b8d22c7fbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:22.424996 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-e3278d4f-c396-437f-8453-89b8d22c7fbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:30:22.427270 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1153/2323] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:30:21 2022] DELETE /identity/v3/projects/cfa7b247fe3b48419a2810ab33a0556b => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:22.429734 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f53b2a0-797b-4b3c-b1fd-b3ea31acb30c None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:22.430973 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f53b2a0-797b-4b3c-b1fd-b3ea31acb30c None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:22.431727 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f53b2a0-797b-4b3c-b1fd-b3ea31acb30c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:22.469325 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-7f53b2a0-797b-4b3c-b1fd-b3ea31acb30c None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:22.472313 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-7f53b2a0-797b-4b3c-b1fd-b3ea31acb30c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:22.517230 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1171/2324] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:30:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:22.521932 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3c42a-2ab4-45e4-b08a-b1b9327fb100 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:22.523118 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3c42a-2ab4-45e4-b08a-b1b9327fb100 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:22.523626 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3c42a-2ab4-45e4-b08a-b1b9327fb100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:22.558480 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-01d3c42a-2ab4-45e4-b08a-b1b9327fb100 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:22.560644 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-01d3c42a-2ab4-45e4-b08a-b1b9327fb100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:22.647444 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1154/2325] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:30:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:22.647444 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f53b2a0-797b-4b3c-b1fd-b3ea31acb30c None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:22.647444 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-7f53b2a0-797b-4b3c-b1fd-b3ea31acb30c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:22.661142 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f53b2a0-797b-4b3c-b1fd-b3ea31acb30c None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:22.676209 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca02200-19c9-4a34-a83d-6b3756b7ee3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:22.677378 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca02200-19c9-4a34-a83d-6b3756b7ee3b admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:22.677849 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca02200-19c9-4a34-a83d-6b3756b7ee3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:22.678313 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca02200-19c9-4a34-a83d-6b3756b7ee3b admin admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:22.679520 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca02200-19c9-4a34-a83d-6b3756b7ee3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:22.684060 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca02200-19c9-4a34-a83d-6b3756b7ee3b admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:22.719694 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1172/2326] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:30:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:22.721396 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07027bdc-3e4a-495e-8787-7ac0d596736c None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:22.721786 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07027bdc-3e4a-495e-8787-7ac0d596736c None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:22.722176 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07027bdc-3e4a-495e-8787-7ac0d596736c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:22.744819 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-07027bdc-3e4a-495e-8787-7ac0d596736c None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:22.746199 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-07027bdc-3e4a-495e-8787-7ac0d596736c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:22.761296 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1173/2327] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:30:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:22.766383 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5808d852-41d5-4b10-b15d-6da66875c861 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:22.766383 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5808d852-41d5-4b10-b15d-6da66875c861 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:22.766383 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5808d852-41d5-4b10-b15d-6da66875c861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:22.791861 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-5808d852-41d5-4b10-b15d-6da66875c861 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:22.791861 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-5808d852-41d5-4b10-b15d-6da66875c861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:22.806141 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1174/2328] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:30:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:22.806703 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5808d852-41d5-4b10-b15d-6da66875c861 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:22.819483 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1155/2329] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:30:22 2022] POST /identity/v3/projects => generated 373 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:30:22.825428 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca02200-19c9-4a34-a83d-6b3756b7ee3b admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:22.825561 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5808d852-41d5-4b10-b15d-6da66875c861 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:22.868964 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:22.927280 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959513ac-b7eb-45f2-8e53-2baa604388fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:22.928503 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959513ac-b7eb-45f2-8e53-2baa604388fe admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:22.928982 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959513ac-b7eb-45f2-8e53-2baa604388fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:22.929478 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959513ac-b7eb-45f2-8e53-2baa604388fe admin admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:22.932676 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959513ac-b7eb-45f2-8e53-2baa604388fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:30:22.932676 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959513ac-b7eb-45f2-8e53-2baa604388fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:22.935673 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959513ac-b7eb-45f2-8e53-2baa604388fe admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:22.940267 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:22.954223 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca02200-19c9-4a34-a83d-6b3756b7ee3b admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:22.960381 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55453666-364a-41e4-8811-f973bc5aad85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:22.961398 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1175/2330] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:30:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:30:22.961970 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55453666-364a-41e4-8811-f973bc5aad85 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:22.962618 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55453666-364a-41e4-8811-f973bc5aad85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:22.962798 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959513ac-b7eb-45f2-8e53-2baa604388fe admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:22.963270 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55453666-364a-41e4-8811-f973bc5aad85 admin admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:22.964271 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55453666-364a-41e4-8811-f973bc5aad85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:30:22.964838 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55453666-364a-41e4-8811-f973bc5aad85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:22.966644 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55453666-364a-41e4-8811-f973bc5aad85 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:22.982244 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959513ac-b7eb-45f2-8e53-2baa604388fe admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:22.990042 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde7533d-34a2-4584-a016-45dbc991f842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:22.990698 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1156/2331] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:30:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:30:22.993189 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55453666-364a-41e4-8811-f973bc5aad85 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:22.994081 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde7533d-34a2-4584-a016-45dbc991f842 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:22.994620 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde7533d-34a2-4584-a016-45dbc991f842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:22.994939 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde7533d-34a2-4584-a016-45dbc991f842 admin admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:22.995698 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde7533d-34a2-4584-a016-45dbc991f842 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:22.997864 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde7533d-34a2-4584-a016-45dbc991f842 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:23.011752 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55453666-364a-41e4-8811-f973bc5aad85 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:23.016786 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca587d-9f63-474d-9699-56b0ade28a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:23.017736 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca587d-9f63-474d-9699-56b0ade28a1b admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:23.017736 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca587d-9f63-474d-9699-56b0ade28a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:23.018008 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca587d-9f63-474d-9699-56b0ade28a1b admin admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:23.018688 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ca587d-9f63-474d-9699-56b0ade28a1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:23.020697 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ca587d-9f63-474d-9699-56b0ade28a1b admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:23.097141 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1176/2332] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:30:22 2022] POST /identity/v3/users => generated 293 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:30:23.098191 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde7533d-34a2-4584-a016-45dbc991f842 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:23.109110 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde7533d-34a2-4584-a016-45dbc991f842 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:23.113968 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35935a1-82ec-449e-8515-fb2053d2b1f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:23.114616 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35935a1-82ec-449e-8515-fb2053d2b1f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:23.114898 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35935a1-82ec-449e-8515-fb2053d2b1f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:23.115199 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35935a1-82ec-449e-8515-fb2053d2b1f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:23.115879 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35935a1-82ec-449e-8515-fb2053d2b1f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:23.119039 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35935a1-82ec-449e-8515-fb2053d2b1f3 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:23.131683 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1157/2333] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:30:22 2022] POST /identity/v3/projects => generated 367 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:30:23.131683 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca587d-9f63-474d-9699-56b0ade28a1b admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:23.203806 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca587d-9f63-474d-9699-56b0ade28a1b admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:23.216251 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da756be-f963-4433-b0b3-3aeea6080628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:23.216251 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da756be-f963-4433-b0b3-3aeea6080628 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:23.216251 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da756be-f963-4433-b0b3-3aeea6080628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:23.216251 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da756be-f963-4433-b0b3-3aeea6080628 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:23.216745 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da756be-f963-4433-b0b3-3aeea6080628 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:23.216836 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1177/2334] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:30:23 2022] POST /identity/v3/projects => generated 385 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:30:23.216937 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da756be-f963-4433-b0b3-3aeea6080628 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:23.219254 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35935a1-82ec-449e-8515-fb2053d2b1f3 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:23.230741 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1158/2335] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:30:23 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:30:23.231926 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da756be-f963-4433-b0b3-3aeea6080628 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:23.243693 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da756be-f963-4433-b0b3-3aeea6080628 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:23.320579 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35935a1-82ec-449e-8515-fb2053d2b1f3 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:23.329974 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9ca8ad-b0a8-4058-a173-323947f6138a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:23.331141 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ca8ad-b0a8-4058-a173-323947f6138a admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:23.331664 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ca8ad-b0a8-4058-a173-323947f6138a admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:23.332134 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ca8ad-b0a8-4058-a173-323947f6138a admin admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:23.340974 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9ca8ad-b0a8-4058-a173-323947f6138a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:23.341407 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8f9555-58a1-4c1b-92a9-ac207aa928af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:23.341407 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f9555-58a1-4c1b-92a9-ac207aa928af admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:23.341407 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f9555-58a1-4c1b-92a9-ac207aa928af admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:23.341407 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f9555-58a1-4c1b-92a9-ac207aa928af admin admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:23.341760 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9ca8ad-b0a8-4058-a173-323947f6138a admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:23.341857 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8f9555-58a1-4c1b-92a9-ac207aa928af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:23.341857 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8f9555-58a1-4c1b-92a9-ac207aa928af admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:23.425569 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1178/2336] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:30:23 2022] POST /identity/v3/users => generated 284 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:30:23.426726 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9ca8ad-b0a8-4058-a173-323947f6138a admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:23.459198 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1159/2337] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:30:23 2022] POST /identity/v3/users => generated 293 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:30:23.460762 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8f9555-58a1-4c1b-92a9-ac207aa928af admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:23.465419 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9ca8ad-b0a8-4058-a173-323947f6138a admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:23.483530 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60366285-90b3-422d-ab6b-4416d1409449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:23.484327 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8f9555-58a1-4c1b-92a9-ac207aa928af admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:23.484518 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60366285-90b3-422d-ab6b-4416d1409449 admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:23.484785 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60366285-90b3-422d-ab6b-4416d1409449 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:23.485071 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60366285-90b3-422d-ab6b-4416d1409449 admin admin] PATH_INFO: `/v3/projects/1a18e7fcbdab4e1b86e201ac4a3f7aa3/users/02a53b7a3b3741378f324fbcebde804a/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:23.491938 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b673f68-ad17-4025-bfb8-84f2ed7269d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:23.493635 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b673f68-ad17-4025-bfb8-84f2ed7269d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:23.494259 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b673f68-ad17-4025-bfb8-84f2ed7269d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:23.494719 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b673f68-ad17-4025-bfb8-84f2ed7269d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:23.496085 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b673f68-ad17-4025-bfb8-84f2ed7269d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:23.501190 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b673f68-ad17-4025-bfb8-84f2ed7269d1 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:23.530466 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1160/2338] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:30:23 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:30:23.531867 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b673f68-ad17-4025-bfb8-84f2ed7269d1 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:23.540180 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60366285-90b3-422d-ab6b-4416d1409449 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a18e7fcbdab4e1b86e201ac4a3f7aa3, user_id=02a53b7a3b3741378f324fbcebde804a, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:23.547393 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60366285-90b3-422d-ab6b-4416d1409449 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:23.562316 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b673f68-ad17-4025-bfb8-84f2ed7269d1 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:23.572844 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82110af7-e733-4780-a171-f2c5f867ebb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:23.574076 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82110af7-e733-4780-a171-f2c5f867ebb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:23.575671 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82110af7-e733-4780-a171-f2c5f867ebb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:23.576177 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82110af7-e733-4780-a171-f2c5f867ebb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:23.577711 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82110af7-e733-4780-a171-f2c5f867ebb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:23.582300 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82110af7-e733-4780-a171-f2c5f867ebb9 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:23.612884 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1161/2339] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:30:23 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:30:23.612884 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82110af7-e733-4780-a171-f2c5f867ebb9 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:23.632079 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82110af7-e733-4780-a171-f2c5f867ebb9 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:23.640563 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1179/2340] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:30:23 2022] PUT /identity/v3/projects/1a18e7fcbdab4e1b86e201ac4a3f7aa3/users/02a53b7a3b3741378f324fbcebde804a/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:23.642340 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60366285-90b3-422d-ab6b-4416d1409449 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:23.687429 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:23.731921 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:23.753660 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60366285-90b3-422d-ab6b-4416d1409449 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:23.755967 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920f8c07-a4e2-45dc-8209-8fab0ffe863a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:23.756808 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f8c07-a4e2-45dc-8209-8fab0ffe863a admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:23.757136 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f8c07-a4e2-45dc-8209-8fab0ffe863a admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:23.757421 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f8c07-a4e2-45dc-8209-8fab0ffe863a admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:23.763023 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920f8c07-a4e2-45dc-8209-8fab0ffe863a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:23.763023 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920f8c07-a4e2-45dc-8209-8fab0ffe863a admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:23.763336 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d805cc0-cd21-482d-85cb-3836c65d94b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:23.763336 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d805cc0-cd21-482d-85cb-3836c65d94b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:23.763336 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d805cc0-cd21-482d-85cb-3836c65d94b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:23.763336 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d805cc0-cd21-482d-85cb-3836c65d94b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:23.763794 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d805cc0-cd21-482d-85cb-3836c65d94b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:23.765193 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d805cc0-cd21-482d-85cb-3836c65d94b5 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:23.773514 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1162/2341] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:30:23 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:30:23.773514 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920f8c07-a4e2-45dc-8209-8fab0ffe863a admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:23.787957 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1180/2342] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:30:23 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:30:23.787957 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d805cc0-cd21-482d-85cb-3836c65d94b5 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:23.791245 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920f8c07-a4e2-45dc-8209-8fab0ffe863a admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:23.798637 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f053c762-8e05-4392-8be7-69641450c029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:23.799897 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053c762-8e05-4392-8be7-69641450c029 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:23.799897 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053c762-8e05-4392-8be7-69641450c029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:23.800341 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053c762-8e05-4392-8be7-69641450c029 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:23.801366 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f053c762-8e05-4392-8be7-69641450c029 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:23.804300 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d805cc0-cd21-482d-85cb-3836c65d94b5 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:23.807560 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f053c762-8e05-4392-8be7-69641450c029 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:23.813583 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a280f-452d-41e4-9acc-d718f318830a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:23.816193 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a280f-452d-41e4-9acc-d718f318830a admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:23.816641 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a280f-452d-41e4-9acc-d718f318830a admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:23.816976 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a280f-452d-41e4-9acc-d718f318830a admin admin] PATH_INFO: `/v3/projects/d655670d534548b08046806bb1f4b283/users/f59c37f1801a473c8ee9c1928856ae26/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:23.839618 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1163/2343] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:30:23 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:30:23.841365 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f053c762-8e05-4392-8be7-69641450c029 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:23.847641 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124a280f-452d-41e4-9acc-d718f318830a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d655670d534548b08046806bb1f4b283, user_id=f59c37f1801a473c8ee9c1928856ae26, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:23.857624 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124a280f-452d-41e4-9acc-d718f318830a admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:23.876765 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f053c762-8e05-4392-8be7-69641450c029 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:23.886010 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e2163d-8341-46c9-8247-a7af9eccd2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:23.892804 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e2163d-8341-46c9-8247-a7af9eccd2e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:23.893266 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e2163d-8341-46c9-8247-a7af9eccd2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:23.893865 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e2163d-8341-46c9-8247-a7af9eccd2e5 admin admin] PATH_INFO: `/v3/projects/1a1786032e9449728e36f679515ced6d/users/0e634ab2f0944e24a087a10e36451f08/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:23.933828 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e2163d-8341-46c9-8247-a7af9eccd2e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a1786032e9449728e36f679515ced6d, user_id=0e634ab2f0944e24a087a10e36451f08, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:23.936725 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e2163d-8341-46c9-8247-a7af9eccd2e5 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:23.959176 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1181/2344] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:30:23 2022] PUT /identity/v3/projects/d655670d534548b08046806bb1f4b283/users/f59c37f1801a473c8ee9c1928856ae26/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:23.959176 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a280f-452d-41e4-9acc-d718f318830a admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:24.093013 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a280f-452d-41e4-9acc-d718f318830a admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:24.093013 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9695cc3b-f39e-4d60-928a-48d4a11728c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:24.093013 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9695cc3b-f39e-4d60-928a-48d4a11728c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:24.093013 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9695cc3b-f39e-4d60-928a-48d4a11728c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:24.093652 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9695cc3b-f39e-4d60-928a-48d4a11728c7 admin admin] PATH_INFO: `/v3/projects/1a18e7fcbdab4e1b86e201ac4a3f7aa3/users/02a53b7a3b3741378f324fbcebde804a/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:24.093652 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9695cc3b-f39e-4d60-928a-48d4a11728c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a18e7fcbdab4e1b86e201ac4a3f7aa3, user_id=02a53b7a3b3741378f324fbcebde804a, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:24.093652 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9695cc3b-f39e-4d60-928a-48d4a11728c7 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:24.108870 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1182/2345] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:30:23 2022] PUT /identity/v3/projects/1a18e7fcbdab4e1b86e201ac4a3f7aa3/users/02a53b7a3b3741378f324fbcebde804a/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:24.115389 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d36b3df-b47c-4e9b-86f1-9f978fd23bf7 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:24.115860 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d36b3df-b47c-4e9b-86f1-9f978fd23bf7 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:24.116147 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d36b3df-b47c-4e9b-86f1-9f978fd23bf7 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:24.118294 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1183/2346] 10.222.0.23 () {54 vars in 921 bytes} [Tue Jun 21 18:30:24 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:30:24.119581 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2550c504-932c-4f5a-8dea-361fcf46bde9 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:24.119820 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2550c504-932c-4f5a-8dea-361fcf46bde9 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:24.120103 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2550c504-932c-4f5a-8dea-361fcf46bde9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:24.156061 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-2550c504-932c-4f5a-8dea-361fcf46bde9 None None] MFA Rules not processed for user `f59c37f1801a473c8ee9c1928856ae26`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:24.157414 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-2550c504-932c-4f5a-8dea-361fcf46bde9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:24.181457 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1164/2347] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:30:23 2022] PUT /identity/v3/projects/1a1786032e9449728e36f679515ced6d/users/0e634ab2f0944e24a087a10e36451f08/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:24.187707 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d03d24-7911-47e4-81f4-5d2ea300d2fa None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:24.187707 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d03d24-7911-47e4-81f4-5d2ea300d2fa None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:24.188163 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d03d24-7911-47e4-81f4-5d2ea300d2fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:24.289954 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1184/2348] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:30:24 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:24.291650 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08cbc2b-6a98-413e-add5-3896c9cb1c7b None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:24.292117 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08cbc2b-6a98-413e-add5-3896c9cb1c7b None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:24.293835 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08cbc2b-6a98-413e-add5-3896c9cb1c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:24.307790 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-09d03d24-7911-47e4-81f4-5d2ea300d2fa None None] MFA Rules not processed for user `904255dd3c1f406eb10f6fc65f9cd098`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:24.311392 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-09d03d24-7911-47e4-81f4-5d2ea300d2fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:24.318982 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-a08cbc2b-6a98-413e-add5-3896c9cb1c7b None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:24.321739 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-a08cbc2b-6a98-413e-add5-3896c9cb1c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:24.488024 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:24.520554 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1165/2349] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Jun 21 18:30:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:24.522740 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d03d24-7911-47e4-81f4-5d2ea300d2fa None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:24.525180 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-09d03d24-7911-47e4-81f4-5d2ea300d2fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:24.556452 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1185/2350] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:30:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:24.557801 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08cbc2b-6a98-413e-add5-3896c9cb1c7b None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:24.562155 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-a08cbc2b-6a98-413e-add5-3896c9cb1c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:24.702497 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:24.764914 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d03d24-7911-47e4-81f4-5d2ea300d2fa None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:24.765234 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08cbc2b-6a98-413e-add5-3896c9cb1c7b None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:24.786117 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54657e22-0c44-4ab6-8d0d-09c719466ef3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9672794bc1ca4dcd8259d51d9290384b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:24.786117 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54657e22-0c44-4ab6-8d0d-09c719466ef3 service glance] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:24.786117 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54657e22-0c44-4ab6-8d0d-09c719466ef3 service glance] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:24.786117 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54657e22-0c44-4ab6-8d0d-09c719466ef3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:24.786955 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4984ae81-9b30-4a61-8b0b-b75b574d663c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9672794bc1ca4dcd8259d51d9290384b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:24.786955 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4984ae81-9b30-4a61-8b0b-b75b574d663c service glance] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:24.786955 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4984ae81-9b30-4a61-8b0b-b75b574d663c service glance] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:24.786955 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4984ae81-9b30-4a61-8b0b-b75b574d663c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:24.805152 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4984ae81-9b30-4a61-8b0b-b75b574d663c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:24.811872 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4984ae81-9b30-4a61-8b0b-b75b574d663c service glance] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:24.850118 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1186/2351] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:30:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:30:24.869954 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4984ae81-9b30-4a61-8b0b-b75b574d663c service glance] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:24.874260 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-4984ae81-9b30-4a61-8b0b-b75b574d663c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:24.967388 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54657e22-0c44-4ab6-8d0d-09c719466ef3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:24.969012 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54657e22-0c44-4ab6-8d0d-09c719466ef3 service glance] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:24.988183 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1166/2352] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:30:24 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:30:25.054796 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4984ae81-9b30-4a61-8b0b-b75b574d663c service glance] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:25.060935 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e819f39b-7732-4a17-9c21-349ca2745acc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:25.062452 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819f39b-7732-4a17-9c21-349ca2745acc service neutron] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:25.063630 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819f39b-7732-4a17-9c21-349ca2745acc service neutron] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:25.064838 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819f39b-7732-4a17-9c21-349ca2745acc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:25.082623 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e819f39b-7732-4a17-9c21-349ca2745acc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:25.084479 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e819f39b-7732-4a17-9c21-349ca2745acc service neutron] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:25.099270 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1187/2353] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:30:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:30:25.255124 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6958fd-5872-4a0e-83b8-3d9ad7c09a5f None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:25.255886 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6958fd-5872-4a0e-83b8-3d9ad7c09a5f None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:25.256364 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6958fd-5872-4a0e-83b8-3d9ad7c09a5f None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:25.260334 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1167/2354] 10.222.0.23 () {54 vars in 921 bytes} [Tue Jun 21 18:30:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:30:25.267799 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5753187f-fe00-4b03-9ed2-1247bed2720d None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:25.268100 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5753187f-fe00-4b03-9ed2-1247bed2720d None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:25.268383 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5753187f-fe00-4b03-9ed2-1247bed2720d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:25.316561 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-5753187f-fe00-4b03-9ed2-1247bed2720d None None] MFA Rules not processed for user `59c7f07e58ba4db9a7d402d8f3da1e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:25.332190 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-5753187f-fe00-4b03-9ed2-1247bed2720d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:25.343735 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1188/2355] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Jun 21 18:30:25 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:30:25.371066 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6958fd-5872-4a0e-83b8-3d9ad7c09a5f None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:25.393876 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6958fd-5872-4a0e-83b8-3d9ad7c09a5f None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:25.404448 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967a10aa-3570-4b47-bfa7-2c51f07f83ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:25.405368 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a10aa-3570-4b47-bfa7-2c51f07f83ad service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:25.407923 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a10aa-3570-4b47-bfa7-2c51f07f83ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:25.408247 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a10aa-3570-4b47-bfa7-2c51f07f83ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:25.424551 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967a10aa-3570-4b47-bfa7-2c51f07f83ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:25.433009 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967a10aa-3570-4b47-bfa7-2c51f07f83ad service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:25.452570 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1168/2356] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:30:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:30:25.807906 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8028231b-f6e4-413b-a306-4f6491a70515 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:25.809759 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8028231b-f6e4-413b-a306-4f6491a70515 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:25.810242 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8028231b-f6e4-413b-a306-4f6491a70515 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:25.812470 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1189/2357] 10.222.0.23 () {54 vars in 921 bytes} [Tue Jun 21 18:30:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:30:25.822371 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de015406-3478-40c1-902b-6a4904cee528 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:25.822659 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de015406-3478-40c1-902b-6a4904cee528 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:25.822940 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de015406-3478-40c1-902b-6a4904cee528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:25.845073 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-de015406-3478-40c1-902b-6a4904cee528 None None] MFA Rules not processed for user `904255dd3c1f406eb10f6fc65f9cd098`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:25.846160 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-de015406-3478-40c1-902b-6a4904cee528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:25.863493 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1169/2358] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Jun 21 18:30:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:25.885898 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8028231b-f6e4-413b-a306-4f6491a70515 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:25.890863 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-8028231b-f6e4-413b-a306-4f6491a70515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:25.922121 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8028231b-f6e4-413b-a306-4f6491a70515 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:25.932738 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d09cdc-01ae-4808-9a94-b814bbac234f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:25.932738 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d09cdc-01ae-4808-9a94-b814bbac234f service neutron] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:25.933337 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d09cdc-01ae-4808-9a94-b814bbac234f service neutron] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:25.935823 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d09cdc-01ae-4808-9a94-b814bbac234f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:25.953072 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d09cdc-01ae-4808-9a94-b814bbac234f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:25.954794 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d09cdc-01ae-4808-9a94-b814bbac234f service neutron] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:25.974047 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1190/2359] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:30:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:30:35.824241 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a2eb3-5ec5-4304-a616-4cd4d8418d09 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:35.824834 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a2eb3-5ec5-4304-a616-4cd4d8418d09 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:35.825519 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a2eb3-5ec5-4304-a616-4cd4d8418d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:35.883852 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-5e3a2eb3-5ec5-4304-a616-4cd4d8418d09 None None] MFA Rules not processed for user `fcfcd9f134e74b09bfdac337c81c8a07`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:35.894009 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-5e3a2eb3-5ec5-4304-a616-4cd4d8418d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:35.966353 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=77 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:36.091867 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1170/2360] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Jun 21 18:30:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:36.127016 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d09cdc-01ae-4808-9a94-b814bbac234f service neutron] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:36.127016 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-32d09cdc-01ae-4808-9a94-b814bbac234f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:36.367498 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d09cdc-01ae-4808-9a94-b814bbac234f service neutron] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:36.389504 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f1a622-6956-4cfe-94cd-8c8de3cad43b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e0d25565f1543d9a7577288fc09119f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:36.390163 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1a622-6956-4cfe-94cd-8c8de3cad43b service swift] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:36.390536 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1a622-6956-4cfe-94cd-8c8de3cad43b service swift] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:36.390816 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1a622-6956-4cfe-94cd-8c8de3cad43b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:36.406612 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f1a622-6956-4cfe-94cd-8c8de3cad43b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:36.407820 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f1a622-6956-4cfe-94cd-8c8de3cad43b service swift] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:36.444896 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1191/2361] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Jun 21 18:30:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 18:30:41.894173 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d711bfe-2cbb-411c-b974-a11c72724e33 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:41.909575 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d711bfe-2cbb-411c-b974-a11c72724e33 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:41.909575 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d711bfe-2cbb-411c-b974-a11c72724e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:42.060709 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-3d711bfe-2cbb-411c-b974-a11c72724e33 None None] MFA Rules not processed for user `0e634ab2f0944e24a087a10e36451f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:42.062383 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-3d711bfe-2cbb-411c-b974-a11c72724e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:42.167270 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:42.181529 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1171/2362] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:30:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:42.183112 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f1a622-6956-4cfe-94cd-8c8de3cad43b service swift] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:42.200810 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad61c194-6701-4e9d-a0ed-11d8bf51a4b0 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:42.201099 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad61c194-6701-4e9d-a0ed-11d8bf51a4b0 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:42.201467 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad61c194-6701-4e9d-a0ed-11d8bf51a4b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:42.210374 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f1a622-6956-4cfe-94cd-8c8de3cad43b service swift] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:42.232536 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9912bd-1855-4aa0-b265-a53c0167d65e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:42.233266 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9912bd-1855-4aa0-b265-a53c0167d65e admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:42.233554 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9912bd-1855-4aa0-b265-a53c0167d65e admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:42.234536 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9912bd-1855-4aa0-b265-a53c0167d65e admin admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:42.234536 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9912bd-1855-4aa0-b265-a53c0167d65e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:42.241047 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9912bd-1855-4aa0-b265-a53c0167d65e admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:42.286414 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-ad61c194-6701-4e9d-a0ed-11d8bf51a4b0 None None] MFA Rules not processed for user `02a53b7a3b3741378f324fbcebde804a`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:42.290376 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-ad61c194-6701-4e9d-a0ed-11d8bf51a4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:42.404732 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1192/2363] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:30:42 2022] POST /identity/v3/projects => generated 385 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:30:42.419267 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9912bd-1855-4aa0-b265-a53c0167d65e admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:42.560357 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1172/2364] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:30:42 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:42.580099 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad61c194-6701-4e9d-a0ed-11d8bf51a4b0 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:42.583852 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-ad61c194-6701-4e9d-a0ed-11d8bf51a4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:42.652469 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad61c194-6701-4e9d-a0ed-11d8bf51a4b0 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:42.700521 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9912bd-1855-4aa0-b265-a53c0167d65e admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:42.725659 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c24e7-9df8-464d-854d-45c76eede014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:42.725659 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c24e7-9df8-464d-854d-45c76eede014 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:42.725659 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c24e7-9df8-464d-854d-45c76eede014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:42.725659 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c24e7-9df8-464d-854d-45c76eede014 admin admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:42.727134 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523c24e7-9df8-464d-854d-45c76eede014 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:42.727134 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523c24e7-9df8-464d-854d-45c76eede014 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:42.950419 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ded9ea3-e86d-4cb3-8e96-583d55e2dc70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904255dd3c1f406eb10f6fc65f9cd098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:42.951679 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded9ea3-e86d-4cb3-8e96-583d55e2dc70 service nova] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:42.953554 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded9ea3-e86d-4cb3-8e96-583d55e2dc70 service nova] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:42.954080 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded9ea3-e86d-4cb3-8e96-583d55e2dc70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:43.137962 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ded9ea3-e86d-4cb3-8e96-583d55e2dc70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:43.137962 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ded9ea3-e86d-4cb3-8e96-583d55e2dc70 service nova] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:43.137962 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1173/2365] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:30:42 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:30:43.570951 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1193/2366] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:30:42 2022] POST /identity/v3/users => generated 299 bytes in 1152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:30:43.574865 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ded9ea3-e86d-4cb3-8e96-583d55e2dc70 service nova] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:43.622275 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ded9ea3-e86d-4cb3-8e96-583d55e2dc70 service nova] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:43.636751 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c030de0a-202a-401f-9cb4-9572233c2a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:43.641107 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c030de0a-202a-401f-9cb4-9572233c2a8d admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:43.641400 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c030de0a-202a-401f-9cb4-9572233c2a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:43.641699 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c030de0a-202a-401f-9cb4-9572233c2a8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:43.642466 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c030de0a-202a-401f-9cb4-9572233c2a8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:43.644216 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c030de0a-202a-401f-9cb4-9572233c2a8d admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:43.689233 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1174/2367] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:30:43.693375 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c24e7-9df8-464d-854d-45c76eede014 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:43.712743 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c24e7-9df8-464d-854d-45c76eede014 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:43.724079 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30002075-6606-4315-8af5-0d8081ced932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:43.734872 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30002075-6606-4315-8af5-0d8081ced932 admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:43.734872 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30002075-6606-4315-8af5-0d8081ced932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:43.734872 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30002075-6606-4315-8af5-0d8081ced932 admin admin] PATH_INFO: `/v3/projects/f8d71f4807a147e4b86450665a2d24a5/users/a043359d5cea4a2c938a1d1c781c8261/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:43.755684 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30002075-6606-4315-8af5-0d8081ced932 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8d71f4807a147e4b86450665a2d24a5, user_id=a043359d5cea4a2c938a1d1c781c8261, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:43.762516 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30002075-6606-4315-8af5-0d8081ced932 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:43.911941 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1194/2368] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:30:43 2022] PUT /identity/v3/projects/f8d71f4807a147e4b86450665a2d24a5/users/a043359d5cea4a2c938a1d1c781c8261/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:43.917466 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c030de0a-202a-401f-9cb4-9572233c2a8d admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:43.936764 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c030de0a-202a-401f-9cb4-9572233c2a8d admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:43.948485 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:44.231096 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0df391b-e20e-44f0-84d7-b222ef3750a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:44.231096 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df391b-e20e-44f0-84d7-b222ef3750a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:44.231096 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df391b-e20e-44f0-84d7-b222ef3750a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:44.232090 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df391b-e20e-44f0-84d7-b222ef3750a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:44.234518 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0df391b-e20e-44f0-84d7-b222ef3750a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:44.238011 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0df391b-e20e-44f0-84d7-b222ef3750a5 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:44.281730 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30002075-6606-4315-8af5-0d8081ced932 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:44.287948 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1175/2369] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:30:44.334614 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30002075-6606-4315-8af5-0d8081ced932 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:44.363169 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02c6c69-f7a9-456f-a102-0cc4bc7eab60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:44.363169 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c6c69-f7a9-456f-a102-0cc4bc7eab60 admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:44.363169 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c6c69-f7a9-456f-a102-0cc4bc7eab60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:44.363169 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c6c69-f7a9-456f-a102-0cc4bc7eab60 admin admin] PATH_INFO: `/v3/projects/f8d71f4807a147e4b86450665a2d24a5/users/a043359d5cea4a2c938a1d1c781c8261/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:44.386004 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02c6c69-f7a9-456f-a102-0cc4bc7eab60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8d71f4807a147e4b86450665a2d24a5, user_id=a043359d5cea4a2c938a1d1c781c8261, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:44.399152 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02c6c69-f7a9-456f-a102-0cc4bc7eab60 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:44.518629 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1195/2370] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:30:44 2022] PUT /identity/v3/projects/f8d71f4807a147e4b86450665a2d24a5/users/a043359d5cea4a2c938a1d1c781c8261/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:30:49.842830 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d8c68d-bd50-4dad-a6c4-aa679ffad753 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:49.845039 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d8c68d-bd50-4dad-a6c4-aa679ffad753 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:49.845039 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d8c68d-bd50-4dad-a6c4-aa679ffad753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:49.908155 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-f7d8c68d-bd50-4dad-a6c4-aa679ffad753 None None] MFA Rules not processed for user `113b17a680c640f19b0f402f371c6cbb`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:49.912592 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-f7d8c68d-bd50-4dad-a6c4-aa679ffad753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:50.133325 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1176/2371] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:30:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:54.444949 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=85,keystone:INSERT=6 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:57.933037 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1830ef6c-50a0-4fbd-8698-9f32c9c99412 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:57.933788 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1830ef6c-50a0-4fbd-8698-9f32c9c99412 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:57.934025 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1830ef6c-50a0-4fbd-8698-9f32c9c99412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:57.970473 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-1830ef6c-50a0-4fbd-8698-9f32c9c99412 None None] MFA Rules not processed for user `a043359d5cea4a2c938a1d1c781c8261`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:57.971888 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-1830ef6c-50a0-4fbd-8698-9f32c9c99412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:58.080454 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1196/2372] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:30:57 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:58.098666 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d8c68d-bd50-4dad-a6c4-aa679ffad753 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:58.177445 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d8c68d-bd50-4dad-a6c4-aa679ffad753 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:58.183407 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d1cb27-cd71-4e1d-8b6a-c37fb0b4ab88 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9672794bc1ca4dcd8259d51d9290384b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:58.184291 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d1cb27-cd71-4e1d-8b6a-c37fb0b4ab88 service glance] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:58.184621 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d1cb27-cd71-4e1d-8b6a-c37fb0b4ab88 service glance] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:58.184884 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d1cb27-cd71-4e1d-8b6a-c37fb0b4ab88 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:58.260897 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d1cb27-cd71-4e1d-8b6a-c37fb0b4ab88 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:58.262945 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d1cb27-cd71-4e1d-8b6a-c37fb0b4ab88 service glance] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:58.274634 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1177/2373] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:30:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:30:58.579618 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca18e54-c88b-48f7-b548-bf888f34a6a7 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:58.579995 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca18e54-c88b-48f7-b548-bf888f34a6a7 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:58.580293 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca18e54-c88b-48f7-b548-bf888f34a6a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:58.609422 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-dca18e54-c88b-48f7-b548-bf888f34a6a7 None None] MFA Rules not processed for user `fcfcd9f134e74b09bfdac337c81c8a07`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:30:58.610998 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-dca18e54-c88b-48f7-b548-bf888f34a6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:30:58.700699 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1197/2374] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Jun 21 18:30:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:30:58.732360 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d1cb27-cd71-4e1d-8b6a-c37fb0b4ab88 service glance] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:30:58.737230 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:30:58.825479 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d1cb27-cd71-4e1d-8b6a-c37fb0b4ab88 service glance] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:30:58.832833 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1958dd2-e1ef-4d79-b265-6e00275b2e1a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e0d25565f1543d9a7577288fc09119f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:30:58.833515 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1958dd2-e1ef-4d79-b265-6e00275b2e1a service swift] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:30:58.833891 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1958dd2-e1ef-4d79-b265-6e00275b2e1a service swift] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:30:58.834208 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1958dd2-e1ef-4d79-b265-6e00275b2e1a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:30:58.847873 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1958dd2-e1ef-4d79-b265-6e00275b2e1a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:30:58.850132 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1958dd2-e1ef-4d79-b265-6e00275b2e1a service swift] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:30:58.874636 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1178/2375] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Jun 21 18:30:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 21 18:31:06.421783 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca18e54-c88b-48f7-b548-bf888f34a6a7 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:06.522489 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca18e54-c88b-48f7-b548-bf888f34a6a7 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:06.527797 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12af622e-9196-48fa-a41b-04235cac5fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:06.528767 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af622e-9196-48fa-a41b-04235cac5fc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:06.529051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af622e-9196-48fa-a41b-04235cac5fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:06.529340 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af622e-9196-48fa-a41b-04235cac5fc2 admin admin] PATH_INFO: `/v3/users/f59c37f1801a473c8ee9c1928856ae26` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:06.530587 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12af622e-9196-48fa-a41b-04235cac5fc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f59c37f1801a473c8ee9c1928856ae26)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:06.532668 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12af622e-9196-48fa-a41b-04235cac5fc2 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:06.732307 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:06.762841 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-12af622e-9196-48fa-a41b-04235cac5fc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f59c37f1801a473c8ee9c1928856ae26'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:06.779012 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-12af622e-9196-48fa-a41b-04235cac5fc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f59c37f1801a473c8ee9c1928856ae26'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:06.865249 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-12af622e-9196-48fa-a41b-04235cac5fc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f59c37f1801a473c8ee9c1928856ae26'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:06.865991 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-12af622e-9196-48fa-a41b-04235cac5fc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f59c37f1801a473c8ee9c1928856ae26'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:06.876013 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-12af622e-9196-48fa-a41b-04235cac5fc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f59c37f1801a473c8ee9c1928856ae26'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:06.884494 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1198/2376] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:31:06 2022] DELETE /identity/v3/users/f59c37f1801a473c8ee9c1928856ae26 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:31:06.893070 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dffed04-0126-4cc8-9cf2-7f9fe66726d2 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:06.893070 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dffed04-0126-4cc8-9cf2-7f9fe66726d2 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:06.893070 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dffed04-0126-4cc8-9cf2-7f9fe66726d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:06.909653 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-8dffed04-0126-4cc8-9cf2-7f9fe66726d2 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:31:06.910935 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-8dffed04-0126-4cc8-9cf2-7f9fe66726d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:06.993883 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1179/2377] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:31:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:31:07.002563 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12af622e-9196-48fa-a41b-04235cac5fc2 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:07.003936 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-12af622e-9196-48fa-a41b-04235cac5fc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:07.076110 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12af622e-9196-48fa-a41b-04235cac5fc2 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:07.080306 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191c1a38-72fe-4d34-b548-97960b784729 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:07.081019 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191c1a38-72fe-4d34-b548-97960b784729 service neutron] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:07.081307 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191c1a38-72fe-4d34-b548-97960b784729 service neutron] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:07.081584 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191c1a38-72fe-4d34-b548-97960b784729 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:07.093372 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191c1a38-72fe-4d34-b548-97960b784729 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:07.095038 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191c1a38-72fe-4d34-b548-97960b784729 service neutron] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:07.106606 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1199/2378] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:31:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:31:07.826702 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dffed04-0126-4cc8-9cf2-7f9fe66726d2 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:07.828892 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-8dffed04-0126-4cc8-9cf2-7f9fe66726d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:07.849342 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dffed04-0126-4cc8-9cf2-7f9fe66726d2 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:07.856892 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fae3b72-a214-4382-be73-7a4bd399f8f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:07.857999 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae3b72-a214-4382-be73-7a4bd399f8f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:07.858300 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae3b72-a214-4382-be73-7a4bd399f8f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:07.858669 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae3b72-a214-4382-be73-7a4bd399f8f2 admin admin] PATH_INFO: `/v3/projects/d655670d534548b08046806bb1f4b283` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:07.860051 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fae3b72-a214-4382-be73-7a4bd399f8f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d655670d534548b08046806bb1f4b283)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:07.862475 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fae3b72-a214-4382-be73-7a4bd399f8f2 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:08.034860 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-8fae3b72-a214-4382-be73-7a4bd399f8f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd655670d534548b08046806bb1f4b283', 'request_id': 'req-8fae3b72-a214-4382-be73-7a4bd399f8f2'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:08.129233 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-8fae3b72-a214-4382-be73-7a4bd399f8f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd655670d534548b08046806bb1f4b283', 'request_id': 'req-8fae3b72-a214-4382-be73-7a4bd399f8f2'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:08.141566 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-8fae3b72-a214-4382-be73-7a4bd399f8f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd655670d534548b08046806bb1f4b283', 'request_id': 'req-8fae3b72-a214-4382-be73-7a4bd399f8f2'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:08.170106 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-8fae3b72-a214-4382-be73-7a4bd399f8f2 admin admin] The token cache is being invalidate because project d655670d534548b08046806bb1f4b283 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:31:08.170106 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-8fae3b72-a214-4382-be73-7a4bd399f8f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:08.170724 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-8fae3b72-a214-4382-be73-7a4bd399f8f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:31:08.172281 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1180/2379] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:31:07 2022] DELETE /identity/v3/projects/d655670d534548b08046806bb1f4b283 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:31:08.427702 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b582b0-f3b3-4fe1-ae0d-7e805615046b None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:08.428329 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b582b0-f3b3-4fe1-ae0d-7e805615046b None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:08.428921 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b582b0-f3b3-4fe1-ae0d-7e805615046b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:08.455813 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-75b582b0-f3b3-4fe1-ae0d-7e805615046b None None] MFA Rules not processed for user `fcfcd9f134e74b09bfdac337c81c8a07`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:31:08.457693 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-75b582b0-f3b3-4fe1-ae0d-7e805615046b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:08.582495 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1200/2380] 10.222.0.23 () {58 vars in 967 bytes} [Tue Jun 21 18:31:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:31:08.613731 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fae3b72-a214-4382-be73-7a4bd399f8f2 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:08.617935 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-8fae3b72-a214-4382-be73-7a4bd399f8f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:08.671082 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:08.804545 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fae3b72-a214-4382-be73-7a4bd399f8f2 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:08.815405 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeaa41c-af20-404f-9295-547b638a66c8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e0d25565f1543d9a7577288fc09119f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:08.816690 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeaa41c-af20-404f-9295-547b638a66c8 service swift] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:08.817273 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeaa41c-af20-404f-9295-547b638a66c8 service swift] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:08.817795 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeaa41c-af20-404f-9295-547b638a66c8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:08.842223 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeaa41c-af20-404f-9295-547b638a66c8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:08.843965 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeaa41c-af20-404f-9295-547b638a66c8 service swift] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:08.856586 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1181/2381] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Jun 21 18:31:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 18:31:12.683341 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f9a8c8-56e3-4bae-b758-2686d63283c3 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:12.683934 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f9a8c8-56e3-4bae-b758-2686d63283c3 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:12.683934 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f9a8c8-56e3-4bae-b758-2686d63283c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:12.703341 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-85f9a8c8-56e3-4bae-b758-2686d63283c3 None None] MFA Rules not processed for user `0e634ab2f0944e24a087a10e36451f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:31:12.704617 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-85f9a8c8-56e3-4bae-b758-2686d63283c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:12.780587 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1201/2382] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:31:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:31:18.696170 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023262d2-6fd6-4a46-b3b4-d053163d0098 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:18.696998 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023262d2-6fd6-4a46-b3b4-d053163d0098 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:18.696998 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023262d2-6fd6-4a46-b3b4-d053163d0098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:18.720754 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-023262d2-6fd6-4a46-b3b4-d053163d0098 None None] MFA Rules not processed for user `fcfcd9f134e74b09bfdac337c81c8a07`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:31:18.722598 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-023262d2-6fd6-4a46-b3b4-d053163d0098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:18.735183 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1182/2383] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Jun 21 18:31:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:31:18.751175 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f9a8c8-56e3-4bae-b758-2686d63283c3 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:18.760048 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:18.770880 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f9a8c8-56e3-4bae-b758-2686d63283c3 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:18.778711 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3aec84-8ff6-40ce-a2b8-b2075de99203 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e0d25565f1543d9a7577288fc09119f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:18.779875 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3aec84-8ff6-40ce-a2b8-b2075de99203 service swift] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:18.780350 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3aec84-8ff6-40ce-a2b8-b2075de99203 service swift] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:18.780882 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3aec84-8ff6-40ce-a2b8-b2075de99203 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:18.791989 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3aec84-8ff6-40ce-a2b8-b2075de99203 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:18.794074 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3aec84-8ff6-40ce-a2b8-b2075de99203 service swift] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:18.818030 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1202/2384] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Jun 21 18:31:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 21 18:31:28.715624 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=28 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:28.970877 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=5 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:43.161462 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023262d2-6fd6-4a46-b3b4-d053163d0098 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:43.164446 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-023262d2-6fd6-4a46-b3b4-d053163d0098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:43.257284 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023262d2-6fd6-4a46-b3b4-d053163d0098 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:43.264522 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c753bd-f170-425b-83ec-fccc06e1f913 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:43.265497 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c753bd-f170-425b-83ec-fccc06e1f913 service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:43.266376 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c753bd-f170-425b-83ec-fccc06e1f913 service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:43.266656 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c753bd-f170-425b-83ec-fccc06e1f913 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:43.269922 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-d8c753bd-f170-425b-83ec-fccc06e1f913 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:43.399702 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c753bd-f170-425b-83ec-fccc06e1f913 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:43.402520 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c753bd-f170-425b-83ec-fccc06e1f913 service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:43.429866 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1183/2385] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:31:43 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:31:50.431655 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f6d3b-f628-47ed-bacd-4e4506803e56 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:50.431655 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f6d3b-f628-47ed-bacd-4e4506803e56 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:50.431655 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f6d3b-f628-47ed-bacd-4e4506803e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:50.449665 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-289f6d3b-f628-47ed-bacd-4e4506803e56 None None] MFA Rules not processed for user `5c21b9f596da493fac827d3573833ed1`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:31:50.450931 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-289f6d3b-f628-47ed-bacd-4e4506803e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:50.549270 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1203/2386] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:31:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:31:53.417553 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=50 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:31:54.727035 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c753bd-f170-425b-83ec-fccc06e1f913 service neutron] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:54.738468 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c753bd-f170-425b-83ec-fccc06e1f913 service neutron] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:54.742899 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0143d655-215f-4674-b699-a58f113daf95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:54.743711 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0143d655-215f-4674-b699-a58f113daf95 service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:54.743848 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0143d655-215f-4674-b699-a58f113daf95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:54.744124 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0143d655-215f-4674-b699-a58f113daf95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:54.746257 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-0143d655-215f-4674-b699-a58f113daf95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:54.851498 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0143d655-215f-4674-b699-a58f113daf95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:54.854344 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0143d655-215f-4674-b699-a58f113daf95 service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:54.874371 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1184/2387] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:31:54 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:31:55.493955 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d4033a-c6af-4886-8870-b7b876c61720 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:55.496096 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d4033a-c6af-4886-8870-b7b876c61720 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:55.496809 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d4033a-c6af-4886-8870-b7b876c61720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:55.530052 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-40d4033a-c6af-4886-8870-b7b876c61720 None None] MFA Rules not processed for user `59f011e91b684e1288dad5f0ac733c0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:31:55.532370 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-40d4033a-c6af-4886-8870-b7b876c61720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:55.644429 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1204/2388] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:31:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:31:58.730933 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0143d655-215f-4674-b699-a58f113daf95 service neutron] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:31:58.734305 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-0143d655-215f-4674-b699-a58f113daf95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:58.758920 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0143d655-215f-4674-b699-a58f113daf95 service neutron] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:31:58.764611 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e571e4-e8a2-4fd7-b2b5-63731ef85588 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:31:58.765292 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e571e4-e8a2-4fd7-b2b5-63731ef85588 service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:31:58.765614 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e571e4-e8a2-4fd7-b2b5-63731ef85588 service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:31:58.765951 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e571e4-e8a2-4fd7-b2b5-63731ef85588 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:31:58.769493 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-51e571e4-e8a2-4fd7-b2b5-63731ef85588 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:31:58.908186 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e571e4-e8a2-4fd7-b2b5-63731ef85588 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:31:58.910099 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e571e4-e8a2-4fd7-b2b5-63731ef85588 service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:31:58.925084 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1185/2389] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:31:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:32:00.448188 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d4033a-c6af-4886-8870-b7b876c61720 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:32:00.461640 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d4033a-c6af-4886-8870-b7b876c61720 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:32:00.481589 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da97a7-432f-4c22-9669-9b37cdd0fcbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:32:00.483896 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da97a7-432f-4c22-9669-9b37cdd0fcbf service neutron] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:32:00.484189 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da97a7-432f-4c22-9669-9b37cdd0fcbf service neutron] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:32:00.484522 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da97a7-432f-4c22-9669-9b37cdd0fcbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:32:00.499951 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-57da97a7-432f-4c22-9669-9b37cdd0fcbf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:32:00.535274 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57da97a7-432f-4c22-9669-9b37cdd0fcbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:32:00.537172 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57da97a7-432f-4c22-9669-9b37cdd0fcbf service neutron] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:32:00.550898 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1205/2390] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:32:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:32:08.219463 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc221e-792d-4eae-ab47-70bf94a1ca9a None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:32:08.220578 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc221e-792d-4eae-ab47-70bf94a1ca9a None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:32:08.220922 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc221e-792d-4eae-ab47-70bf94a1ca9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:32:08.245543 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-9ebc221e-792d-4eae-ab47-70bf94a1ca9a None None] MFA Rules not processed for user `e03c3a73166b4bbe9fca6d18ecf59221`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:32:08.247308 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-9ebc221e-792d-4eae-ab47-70bf94a1ca9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:32:08.360056 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1186/2391] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:32:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:32:10.544682 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=56 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:32:17.619083 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da97a7-432f-4c22-9669-9b37cdd0fcbf service neutron] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:32:17.620698 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-57da97a7-432f-4c22-9669-9b37cdd0fcbf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:32:17.636913 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da97a7-432f-4c22-9669-9b37cdd0fcbf service neutron] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:32:17.643035 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a76dd5-c09c-44d9-8a10-e38e4d3bfd0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904255dd3c1f406eb10f6fc65f9cd098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:32:17.643712 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a76dd5-c09c-44d9-8a10-e38e4d3bfd0a service nova] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:32:17.644537 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a76dd5-c09c-44d9-8a10-e38e4d3bfd0a service nova] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:32:17.644537 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a76dd5-c09c-44d9-8a10-e38e4d3bfd0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:32:17.646259 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-a7a76dd5-c09c-44d9-8a10-e38e4d3bfd0a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:32:17.658464 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a76dd5-c09c-44d9-8a10-e38e4d3bfd0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:32:17.660183 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a76dd5-c09c-44d9-8a10-e38e4d3bfd0a service nova] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:32:17.671414 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1206/2392] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:32:17 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:32:18.340356 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=89 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:32:27.666597 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=6 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:32:44.027964 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc221e-792d-4eae-ab47-70bf94a1ca9a None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:32:44.029552 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-9ebc221e-792d-4eae-ab47-70bf94a1ca9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:32:44.095808 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc221e-792d-4eae-ab47-70bf94a1ca9a None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:32:44.099488 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b40c922-0f4a-4f15-9dc5-33f7c0004cb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:32:44.100176 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b40c922-0f4a-4f15-9dc5-33f7c0004cb0 service cinder] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:32:44.100517 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b40c922-0f4a-4f15-9dc5-33f7c0004cb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:32:44.100804 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b40c922-0f4a-4f15-9dc5-33f7c0004cb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:32:44.103073 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-5b40c922-0f4a-4f15-9dc5-33f7c0004cb0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:32:44.115148 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b40c922-0f4a-4f15-9dc5-33f7c0004cb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:32:44.116642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b40c922-0f4a-4f15-9dc5-33f7c0004cb0 service cinder] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:32:44.127264 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1187/2393] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:32:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:32:51.144742 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a76dd5-c09c-44d9-8a10-e38e4d3bfd0a service nova] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:32:51.146232 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-a7a76dd5-c09c-44d9-8a10-e38e4d3bfd0a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:32:51.161165 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a76dd5-c09c-44d9-8a10-e38e4d3bfd0a service nova] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:32:51.168566 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049890ba-605d-4f8f-9dd3-be27425387c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904255dd3c1f406eb10f6fc65f9cd098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:32:51.169264 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049890ba-605d-4f8f-9dd3-be27425387c5 service nova] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:32:51.170107 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049890ba-605d-4f8f-9dd3-be27425387c5 service nova] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:32:51.170107 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049890ba-605d-4f8f-9dd3-be27425387c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:32:51.171930 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-049890ba-605d-4f8f-9dd3-be27425387c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:32:51.186250 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049890ba-605d-4f8f-9dd3-be27425387c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:32:51.189767 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049890ba-605d-4f8f-9dd3-be27425387c5 service nova] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:32:51.207718 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1207/2394] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:32:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:32:52.561989 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b40c922-0f4a-4f15-9dc5-33f7c0004cb0 service cinder] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:32:52.576498 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b40c922-0f4a-4f15-9dc5-33f7c0004cb0 service cinder] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:32:52.582542 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7b0e79-7633-4bf5-b698-e2f5a0da3fe0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904255dd3c1f406eb10f6fc65f9cd098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:32:52.583252 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b0e79-7633-4bf5-b698-e2f5a0da3fe0 service nova] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:32:52.583774 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b0e79-7633-4bf5-b698-e2f5a0da3fe0 service nova] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:32:52.583891 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b0e79-7633-4bf5-b698-e2f5a0da3fe0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:32:52.585784 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-da7b0e79-7633-4bf5-b698-e2f5a0da3fe0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:32:52.614048 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7b0e79-7633-4bf5-b698-e2f5a0da3fe0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:32:52.614048 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7b0e79-7633-4bf5-b698-e2f5a0da3fe0 service nova] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:32:52.621619 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1188/2395] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:32:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:33:01.200934 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=6 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:02.616828 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=32 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:09.131257 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049890ba-605d-4f8f-9dd3-be27425387c5 service nova] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:09.148746 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049890ba-605d-4f8f-9dd3-be27425387c5 service nova] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:09.159289 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b943268-2e90-4b01-bd0e-b6ef8b32dd92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904255dd3c1f406eb10f6fc65f9cd098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:09.160437 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b943268-2e90-4b01-bd0e-b6ef8b32dd92 service nova] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:09.161372 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b943268-2e90-4b01-bd0e-b6ef8b32dd92 service nova] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:09.161832 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b943268-2e90-4b01-bd0e-b6ef8b32dd92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:09.164272 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-0b943268-2e90-4b01-bd0e-b6ef8b32dd92 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:09.180549 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b943268-2e90-4b01-bd0e-b6ef8b32dd92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:09.183705 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b943268-2e90-4b01-bd0e-b6ef8b32dd92 service nova] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:09.201970 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1208/2396] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:33:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:33:09.308119 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7b0e79-7633-4bf5-b698-e2f5a0da3fe0 service nova] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:09.325914 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7b0e79-7633-4bf5-b698-e2f5a0da3fe0 service nova] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:09.328528 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48be4b80-d5e0-4ee2-9f92-0e31af7f77b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:09.329349 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be4b80-d5e0-4ee2-9f92-0e31af7f77b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:09.329917 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be4b80-d5e0-4ee2-9f92-0e31af7f77b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:09.330298 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be4b80-d5e0-4ee2-9f92-0e31af7f77b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:09.332431 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-48be4b80-d5e0-4ee2-9f92-0e31af7f77b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:09.426522 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48be4b80-d5e0-4ee2-9f92-0e31af7f77b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:09.428873 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48be4b80-d5e0-4ee2-9f92-0e31af7f77b4 service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:09.442358 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1189/2397] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:33:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:33:14.926899 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b943268-2e90-4b01-bd0e-b6ef8b32dd92 service nova] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:14.930553 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-0b943268-2e90-4b01-bd0e-b6ef8b32dd92 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:14.973065 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b943268-2e90-4b01-bd0e-b6ef8b32dd92 service nova] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:14.983043 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55529ed-055c-4898-b184-631bca480f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:14.984722 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55529ed-055c-4898-b184-631bca480f37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:14.985373 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55529ed-055c-4898-b184-631bca480f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:14.985862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55529ed-055c-4898-b184-631bca480f37 admin admin] PATH_INFO: `/v3/users/59f011e91b684e1288dad5f0ac733c0f` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:14.988622 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55529ed-055c-4898-b184-631bca480f37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59f011e91b684e1288dad5f0ac733c0f)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:14.992855 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55529ed-055c-4898-b184-631bca480f37 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:15.265791 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-c55529ed-055c-4898-b184-631bca480f37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59f011e91b684e1288dad5f0ac733c0f'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:15.291821 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-c55529ed-055c-4898-b184-631bca480f37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59f011e91b684e1288dad5f0ac733c0f'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:15.387758 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-c55529ed-055c-4898-b184-631bca480f37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59f011e91b684e1288dad5f0ac733c0f'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:15.389358 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-c55529ed-055c-4898-b184-631bca480f37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59f011e91b684e1288dad5f0ac733c0f'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:15.411309 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-c55529ed-055c-4898-b184-631bca480f37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59f011e91b684e1288dad5f0ac733c0f'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:15.432007 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1209/2398] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:33:14 2022] DELETE /identity/v3/users/59f011e91b684e1288dad5f0ac733c0f => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:33:15.843347 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a2afe-2f46-40e8-8990-ee4e2a0d926d None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:15.844071 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a2afe-2f46-40e8-8990-ee4e2a0d926d None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:15.844295 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a2afe-2f46-40e8-8990-ee4e2a0d926d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:15.859618 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55529ed-055c-4898-b184-631bca480f37 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:15.866950 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-c55529ed-055c-4898-b184-631bca480f37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:15.900725 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-e16a2afe-2f46-40e8-8990-ee4e2a0d926d None None] MFA Rules not processed for user `5c21b9f596da493fac827d3573833ed1`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:33:15.904063 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-e16a2afe-2f46-40e8-8990-ee4e2a0d926d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:16.038450 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55529ed-055c-4898-b184-631bca480f37 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:16.038450 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e40713a-01ab-433f-be10-01eecb9bf6a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:16.038450 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e40713a-01ab-433f-be10-01eecb9bf6a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:16.038450 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e40713a-01ab-433f-be10-01eecb9bf6a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:16.039477 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e40713a-01ab-433f-be10-01eecb9bf6a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:16.044594 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-1e40713a-01ab-433f-be10-01eecb9bf6a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:16.057354 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e40713a-01ab-433f-be10-01eecb9bf6a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:16.059234 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e40713a-01ab-433f-be10-01eecb9bf6a8 service neutron] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:16.072287 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1190/2399] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:33:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:33:16.072453 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1210/2400] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:33:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:33:16.072569 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16a2afe-2f46-40e8-8990-ee4e2a0d926d None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:16.075151 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-e16a2afe-2f46-40e8-8990-ee4e2a0d926d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:16.242649 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16a2afe-2f46-40e8-8990-ee4e2a0d926d None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:16.253333 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cdd3d4-c136-462c-bc9f-3752366ccb77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:16.254144 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cdd3d4-c136-462c-bc9f-3752366ccb77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:16.254434 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cdd3d4-c136-462c-bc9f-3752366ccb77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:16.255698 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cdd3d4-c136-462c-bc9f-3752366ccb77 admin admin] PATH_INFO: `/v3/projects/10aedc59673c405096de01eaf61ecd6f` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:16.259693 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cdd3d4-c136-462c-bc9f-3752366ccb77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10aedc59673c405096de01eaf61ecd6f)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:16.269954 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cdd3d4-c136-462c-bc9f-3752366ccb77 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:16.476881 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:16.499448 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-c0cdd3d4-c136-462c-bc9f-3752366ccb77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10aedc59673c405096de01eaf61ecd6f', 'request_id': 'req-c0cdd3d4-c136-462c-bc9f-3752366ccb77'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:16.619575 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-c0cdd3d4-c136-462c-bc9f-3752366ccb77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10aedc59673c405096de01eaf61ecd6f', 'request_id': 'req-c0cdd3d4-c136-462c-bc9f-3752366ccb77'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:16.632521 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-c0cdd3d4-c136-462c-bc9f-3752366ccb77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10aedc59673c405096de01eaf61ecd6f', 'request_id': 'req-c0cdd3d4-c136-462c-bc9f-3752366ccb77'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:16.647843 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-c0cdd3d4-c136-462c-bc9f-3752366ccb77 admin admin] The token cache is being invalidate because project 10aedc59673c405096de01eaf61ecd6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:33:16.648307 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-c0cdd3d4-c136-462c-bc9f-3752366ccb77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:16.649255 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-c0cdd3d4-c136-462c-bc9f-3752366ccb77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:16.655642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1191/2401] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:33:16 2022] DELETE /identity/v3/projects/10aedc59673c405096de01eaf61ecd6f => generated 0 bytes in 580 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:33:16.657387 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78eefe8-306e-43ad-b79d-3c8c69754518 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:16.657670 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78eefe8-306e-43ad-b79d-3c8c69754518 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:16.657951 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78eefe8-306e-43ad-b79d-3c8c69754518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:16.694661 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-f78eefe8-306e-43ad-b79d-3c8c69754518 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:33:16.697037 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-f78eefe8-306e-43ad-b79d-3c8c69754518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:16.802050 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1211/2402] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:33:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:33:16.811119 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cdd3d4-c136-462c-bc9f-3752366ccb77 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:16.814279 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-c0cdd3d4-c136-462c-bc9f-3752366ccb77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:16.818582 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d97f51-dada-4c3d-951a-71120e699f5b None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:16.819219 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d97f51-dada-4c3d-951a-71120e699f5b None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:16.819908 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d97f51-dada-4c3d-951a-71120e699f5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:16.833534 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:16.855193 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-e1d97f51-dada-4c3d-951a-71120e699f5b None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:33:16.856475 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-e1d97f51-dada-4c3d-951a-71120e699f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:16.871512 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1212/2403] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:33:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:33:16.872873 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d97f51-dada-4c3d-951a-71120e699f5b None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:16.887203 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d97f51-dada-4c3d-951a-71120e699f5b None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:16.890362 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61947cbe-385d-44e2-b6c1-e614bb2994a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:16.891151 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61947cbe-385d-44e2-b6c1-e614bb2994a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:16.891452 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61947cbe-385d-44e2-b6c1-e614bb2994a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:16.891657 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61947cbe-385d-44e2-b6c1-e614bb2994a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:16.892423 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61947cbe-385d-44e2-b6c1-e614bb2994a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:33:16.892789 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61947cbe-385d-44e2-b6c1-e614bb2994a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:16.894450 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61947cbe-385d-44e2-b6c1-e614bb2994a2 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:16.915533 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1213/2404] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:33:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:33:16.918075 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61947cbe-385d-44e2-b6c1-e614bb2994a2 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:16.942858 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61947cbe-385d-44e2-b6c1-e614bb2994a2 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:16.959413 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4227e5b-1d8d-4a87-89a7-20c39372225d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:16.961246 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4227e5b-1d8d-4a87-89a7-20c39372225d admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:16.962011 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4227e5b-1d8d-4a87-89a7-20c39372225d admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:16.963156 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4227e5b-1d8d-4a87-89a7-20c39372225d admin admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:16.964701 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cdd3d4-c136-462c-bc9f-3752366ccb77 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:16.966355 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4227e5b-1d8d-4a87-89a7-20c39372225d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:16.970144 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb35e0e-83f4-4173-b48a-b37120dfbf53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:16.971215 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb35e0e-83f4-4173-b48a-b37120dfbf53 service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:16.971692 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4227e5b-1d8d-4a87-89a7-20c39372225d admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:16.971897 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb35e0e-83f4-4173-b48a-b37120dfbf53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:16.972180 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb35e0e-83f4-4173-b48a-b37120dfbf53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:16.975089 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-5eb35e0e-83f4-4173-b48a-b37120dfbf53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:16.997772 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb35e0e-83f4-4173-b48a-b37120dfbf53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:17.000594 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb35e0e-83f4-4173-b48a-b37120dfbf53 service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:17.023030 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1192/2405] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:33:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:33:17.150754 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1214/2406] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:33:16 2022] POST /identity/v3/projects => generated 373 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:33:17.156661 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb35e0e-83f4-4173-b48a-b37120dfbf53 service neutron] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:17.175397 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb35e0e-83f4-4173-b48a-b37120dfbf53 service neutron] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:17.256766 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38db141-bfed-4453-aa83-e12ce19daa09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:17.257123 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38db141-bfed-4453-aa83-e12ce19daa09 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:17.257444 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38db141-bfed-4453-aa83-e12ce19daa09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:17.257722 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38db141-bfed-4453-aa83-e12ce19daa09 admin admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:17.258469 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38db141-bfed-4453-aa83-e12ce19daa09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:17.261926 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38db141-bfed-4453-aa83-e12ce19daa09 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:17.377064 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1193/2407] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:33:17 2022] POST /identity/v3/users => generated 287 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:33:17.383049 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4227e5b-1d8d-4a87-89a7-20c39372225d admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:17.404223 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4227e5b-1d8d-4a87-89a7-20c39372225d admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:17.413029 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad24ba9-1955-4004-a352-18091c1ffbbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:17.414223 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad24ba9-1955-4004-a352-18091c1ffbbe admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:17.414726 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad24ba9-1955-4004-a352-18091c1ffbbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:17.415858 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad24ba9-1955-4004-a352-18091c1ffbbe admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:17.419001 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad24ba9-1955-4004-a352-18091c1ffbbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:17.422939 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad24ba9-1955-4004-a352-18091c1ffbbe admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:17.451756 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1215/2408] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:33:17 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:33:17.454760 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38db141-bfed-4453-aa83-e12ce19daa09 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:17.475232 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38db141-bfed-4453-aa83-e12ce19daa09 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:17.485978 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65fc834-a7bd-415e-bfad-496e11f67581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:17.487192 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65fc834-a7bd-415e-bfad-496e11f67581 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:17.487698 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65fc834-a7bd-415e-bfad-496e11f67581 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:17.488207 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65fc834-a7bd-415e-bfad-496e11f67581 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:17.489743 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65fc834-a7bd-415e-bfad-496e11f67581 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:17.492822 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65fc834-a7bd-415e-bfad-496e11f67581 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:17.516625 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1194/2409] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:33:17 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:33:17.522547 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad24ba9-1955-4004-a352-18091c1ffbbe admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:17.534236 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad24ba9-1955-4004-a352-18091c1ffbbe admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:17.540222 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d78c12-c411-4716-aaf4-e9024eb1c19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:17.542258 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d78c12-c411-4716-aaf4-e9024eb1c19e admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:17.542659 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d78c12-c411-4716-aaf4-e9024eb1c19e admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:17.543047 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d78c12-c411-4716-aaf4-e9024eb1c19e admin admin] PATH_INFO: `/v3/projects/fa622e06228e4136921d18ef9dacd5a1/users/7a5dc3889cfc4418bf8447b7088f0572/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:17.579825 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d78c12-c411-4716-aaf4-e9024eb1c19e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa622e06228e4136921d18ef9dacd5a1, user_id=7a5dc3889cfc4418bf8447b7088f0572, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:17.585480 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d78c12-c411-4716-aaf4-e9024eb1c19e admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:17.671741 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1216/2410] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:33:17 2022] PUT /identity/v3/projects/fa622e06228e4136921d18ef9dacd5a1/users/7a5dc3889cfc4418bf8447b7088f0572/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:33:17.689626 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e6955-67a3-42b4-84de-ee3fc60454c1 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:17.690122 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e6955-67a3-42b4-84de-ee3fc60454c1 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:17.690615 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e6955-67a3-42b4-84de-ee3fc60454c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:17.739162 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-820e6955-67a3-42b4-84de-ee3fc60454c1 None None] MFA Rules not processed for user `7a5dc3889cfc4418bf8447b7088f0572`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:33:17.739642 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-820e6955-67a3-42b4-84de-ee3fc60454c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:17.823139 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:17.866852 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1195/2411] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:33:17 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:33:18.021830 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d78c12-c411-4716-aaf4-e9024eb1c19e admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:18.024860 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-21d78c12-c411-4716-aaf4-e9024eb1c19e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:18.036811 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820e6955-67a3-42b4-84de-ee3fc60454c1 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:18.045060 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-820e6955-67a3-42b4-84de-ee3fc60454c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:18.179385 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d78c12-c411-4716-aaf4-e9024eb1c19e admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:18.199930 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01f2db1-d883-43c1-bb7d-280d635f84ce service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938187967efe49be9c5389d1feef81ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:18.200639 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01f2db1-d883-43c1-bb7d-280d635f84ce service placement] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:18.200933 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01f2db1-d883-43c1-bb7d-280d635f84ce service placement] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:18.201213 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01f2db1-d883-43c1-bb7d-280d635f84ce service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:18.203660 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-a01f2db1-d883-43c1-bb7d-280d635f84ce service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:18.223095 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01f2db1-d883-43c1-bb7d-280d635f84ce service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:18.223596 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01f2db1-d883-43c1-bb7d-280d635f84ce service placement] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:18.225736 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820e6955-67a3-42b4-84de-ee3fc60454c1 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:18.242437 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd6a10e-5ac5-422f-a8c2-c81b3ad19230 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904255dd3c1f406eb10f6fc65f9cd098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:18.242437 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6a10e-5ac5-422f-a8c2-c81b3ad19230 service nova] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:18.242437 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6a10e-5ac5-422f-a8c2-c81b3ad19230 service nova] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:18.242792 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6a10e-5ac5-422f-a8c2-c81b3ad19230 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:18.253346 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1217/2412] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:33:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:33:18.258945 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd6a10e-5ac5-422f-a8c2-c81b3ad19230 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:18.261441 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd6a10e-5ac5-422f-a8c2-c81b3ad19230 service nova] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:18.277317 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1196/2413] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:33:18 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:33:18.614738 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01f2db1-d883-43c1-bb7d-280d635f84ce service placement] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:18.769029 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01f2db1-d883-43c1-bb7d-280d635f84ce service placement] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:18.788861 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76afe8c-4041-424b-98de-c88eb7647680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:18.788861 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76afe8c-4041-424b-98de-c88eb7647680 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:18.788861 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76afe8c-4041-424b-98de-c88eb7647680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:18.788861 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76afe8c-4041-424b-98de-c88eb7647680 admin admin] PATH_INFO: `/v3/users/7a5dc3889cfc4418bf8447b7088f0572` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:18.789365 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76afe8c-4041-424b-98de-c88eb7647680 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a5dc3889cfc4418bf8447b7088f0572)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:18.789365 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76afe8c-4041-424b-98de-c88eb7647680 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:18.799088 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:19.117111 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-a76afe8c-4041-424b-98de-c88eb7647680 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a5dc3889cfc4418bf8447b7088f0572'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:19.142937 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-a76afe8c-4041-424b-98de-c88eb7647680 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a5dc3889cfc4418bf8447b7088f0572'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:19.229482 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-a76afe8c-4041-424b-98de-c88eb7647680 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a5dc3889cfc4418bf8447b7088f0572'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:19.230523 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-a76afe8c-4041-424b-98de-c88eb7647680 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a5dc3889cfc4418bf8447b7088f0572'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:19.256590 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-a76afe8c-4041-424b-98de-c88eb7647680 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a5dc3889cfc4418bf8447b7088f0572'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:19.272663 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1218/2414] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:33:18 2022] DELETE /identity/v3/users/7a5dc3889cfc4418bf8447b7088f0572 => generated 0 bytes in 655 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:33:19.274533 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5139d3-be9f-427c-9fab-a2345c222235 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:19.275776 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5139d3-be9f-427c-9fab-a2345c222235 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:19.275970 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5139d3-be9f-427c-9fab-a2345c222235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:19.304294 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-4b5139d3-be9f-427c-9fab-a2345c222235 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:33:19.309011 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-4b5139d3-be9f-427c-9fab-a2345c222235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:19.436561 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1197/2415] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:33:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:33:19.444196 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76afe8c-4041-424b-98de-c88eb7647680 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:19.445827 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-a76afe8c-4041-424b-98de-c88eb7647680 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:19.535393 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76afe8c-4041-424b-98de-c88eb7647680 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:19.541686 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0835c41-298e-4143-b927-30cdf0b35468 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:19.542816 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0835c41-298e-4143-b927-30cdf0b35468 service neutron] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:19.543193 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0835c41-298e-4143-b927-30cdf0b35468 service neutron] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:19.543610 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0835c41-298e-4143-b927-30cdf0b35468 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:19.562466 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0835c41-298e-4143-b927-30cdf0b35468 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:19.565327 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0835c41-298e-4143-b927-30cdf0b35468 service neutron] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:19.582363 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1219/2416] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:33:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:33:20.194896 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5139d3-be9f-427c-9fab-a2345c222235 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:20.197058 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-4b5139d3-be9f-427c-9fab-a2345c222235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:20.215643 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5139d3-be9f-427c-9fab-a2345c222235 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:20.221976 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2140dfba-7afb-4481-b152-9b34af9f43d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:20.223135 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2140dfba-7afb-4481-b152-9b34af9f43d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:20.223481 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2140dfba-7afb-4481-b152-9b34af9f43d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:20.223840 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2140dfba-7afb-4481-b152-9b34af9f43d0 admin admin] PATH_INFO: `/v3/projects/fa622e06228e4136921d18ef9dacd5a1` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:20.226007 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2140dfba-7afb-4481-b152-9b34af9f43d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa622e06228e4136921d18ef9dacd5a1)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:20.230810 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2140dfba-7afb-4481-b152-9b34af9f43d0 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:20.405363 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-2140dfba-7afb-4481-b152-9b34af9f43d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa622e06228e4136921d18ef9dacd5a1', 'request_id': 'req-2140dfba-7afb-4481-b152-9b34af9f43d0'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:20.517501 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-2140dfba-7afb-4481-b152-9b34af9f43d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa622e06228e4136921d18ef9dacd5a1', 'request_id': 'req-2140dfba-7afb-4481-b152-9b34af9f43d0'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:20.522924 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:20.525889 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-2140dfba-7afb-4481-b152-9b34af9f43d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa622e06228e4136921d18ef9dacd5a1', 'request_id': 'req-2140dfba-7afb-4481-b152-9b34af9f43d0'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:20.539335 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-2140dfba-7afb-4481-b152-9b34af9f43d0 admin admin] The token cache is being invalidate because project fa622e06228e4136921d18ef9dacd5a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:33:20.539714 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-2140dfba-7afb-4481-b152-9b34af9f43d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:20.541254 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-2140dfba-7afb-4481-b152-9b34af9f43d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:20.542898 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1198/2417] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:33:20 2022] DELETE /identity/v3/projects/fa622e06228e4136921d18ef9dacd5a1 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:33:20.548290 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af134b3-8e02-4fef-ad77-ab3f3c384627 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:20.548959 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af134b3-8e02-4fef-ad77-ab3f3c384627 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:20.548959 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af134b3-8e02-4fef-ad77-ab3f3c384627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:20.569668 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-4af134b3-8e02-4fef-ad77-ab3f3c384627 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:33:20.570908 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-4af134b3-8e02-4fef-ad77-ab3f3c384627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:20.655708 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1220/2418] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:33:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:33:20.663002 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73856285-f7dc-4e9b-9a46-9b9414d39fb6 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:20.664370 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73856285-f7dc-4e9b-9a46-9b9414d39fb6 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:20.664370 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73856285-f7dc-4e9b-9a46-9b9414d39fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:20.695848 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-73856285-f7dc-4e9b-9a46-9b9414d39fb6 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:33:20.698026 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-73856285-f7dc-4e9b-9a46-9b9414d39fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:20.708117 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1199/2419] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:33:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:33:20.720370 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af134b3-8e02-4fef-ad77-ab3f3c384627 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:20.735834 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af134b3-8e02-4fef-ad77-ab3f3c384627 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:20.741404 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2309bdd-27df-4e5e-8d4f-22958691074c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:20.747112 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2309bdd-27df-4e5e-8d4f-22958691074c admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:20.747112 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2309bdd-27df-4e5e-8d4f-22958691074c admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:20.747112 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2309bdd-27df-4e5e-8d4f-22958691074c admin admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:20.747112 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2309bdd-27df-4e5e-8d4f-22958691074c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:33:20.747112 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2309bdd-27df-4e5e-8d4f-22958691074c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:20.747825 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2309bdd-27df-4e5e-8d4f-22958691074c admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:20.763927 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1221/2420] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:33:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:33:20.768288 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73856285-f7dc-4e9b-9a46-9b9414d39fb6 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:20.779066 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73856285-f7dc-4e9b-9a46-9b9414d39fb6 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:20.786030 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e127c8-d75c-46a6-8f5c-65bea6e1f2f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:20.787329 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e127c8-d75c-46a6-8f5c-65bea6e1f2f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:20.787843 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e127c8-d75c-46a6-8f5c-65bea6e1f2f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:20.788294 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e127c8-d75c-46a6-8f5c-65bea6e1f2f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:20.789175 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e127c8-d75c-46a6-8f5c-65bea6e1f2f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:20.792697 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e127c8-d75c-46a6-8f5c-65bea6e1f2f6 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:20.904752 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1200/2421] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:33:20 2022] POST /identity/v3/projects => generated 365 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:33:20.909963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2309bdd-27df-4e5e-8d4f-22958691074c admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:20.936946 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2309bdd-27df-4e5e-8d4f-22958691074c admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:20.960169 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:21.051887 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20b6290-bad3-4ef8-9544-74c720b63abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:21.053119 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20b6290-bad3-4ef8-9544-74c720b63abe admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:21.053656 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20b6290-bad3-4ef8-9544-74c720b63abe admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:21.054134 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20b6290-bad3-4ef8-9544-74c720b63abe admin admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:21.055473 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20b6290-bad3-4ef8-9544-74c720b63abe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:21.061172 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20b6290-bad3-4ef8-9544-74c720b63abe admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:21.197532 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1222/2422] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:33:20 2022] POST /identity/v3/users => generated 283 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:33:21.201598 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e127c8-d75c-46a6-8f5c-65bea6e1f2f6 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:21.219723 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e127c8-d75c-46a6-8f5c-65bea6e1f2f6 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:21.225054 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8499dd61-d7ef-475d-a36b-b946b49333a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:21.229980 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8499dd61-d7ef-475d-a36b-b946b49333a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:21.230629 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8499dd61-d7ef-475d-a36b-b946b49333a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:21.231316 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8499dd61-d7ef-475d-a36b-b946b49333a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:21.232777 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8499dd61-d7ef-475d-a36b-b946b49333a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:21.235696 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8499dd61-d7ef-475d-a36b-b946b49333a3 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:21.257681 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1201/2423] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:33:21 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:33:21.263012 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20b6290-bad3-4ef8-9544-74c720b63abe admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:21.277032 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20b6290-bad3-4ef8-9544-74c720b63abe admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:21.282929 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce298ac-cb1f-4869-9279-a8356502a84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:21.283633 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce298ac-cb1f-4869-9279-a8356502a84a admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:21.283924 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce298ac-cb1f-4869-9279-a8356502a84a admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:21.284193 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce298ac-cb1f-4869-9279-a8356502a84a admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:21.285103 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce298ac-cb1f-4869-9279-a8356502a84a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:21.287844 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce298ac-cb1f-4869-9279-a8356502a84a admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:21.301305 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1223/2424] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:33:21 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:33:21.307986 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8499dd61-d7ef-475d-a36b-b946b49333a3 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:21.329147 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8499dd61-d7ef-475d-a36b-b946b49333a3 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:21.342639 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0652911d-8d36-4e50-bc7b-75a7b8c25c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:21.344274 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0652911d-8d36-4e50-bc7b-75a7b8c25c69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:21.344792 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0652911d-8d36-4e50-bc7b-75a7b8c25c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:21.345274 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0652911d-8d36-4e50-bc7b-75a7b8c25c69 admin admin] PATH_INFO: `/v3/projects/92ea2d080d0849e29ee7db503c1ee07d/users/db3c1a9e5517403598d1278a14bbab24/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:21.366416 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0652911d-8d36-4e50-bc7b-75a7b8c25c69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92ea2d080d0849e29ee7db503c1ee07d, user_id=db3c1a9e5517403598d1278a14bbab24, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:21.370207 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0652911d-8d36-4e50-bc7b-75a7b8c25c69 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:21.513909 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1202/2425] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:33:21 2022] PUT /identity/v3/projects/92ea2d080d0849e29ee7db503c1ee07d/users/db3c1a9e5517403598d1278a14bbab24/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:33:21.516925 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad1ebc4-7a5a-4638-89ec-fa0a6e8a9894 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:21.517463 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad1ebc4-7a5a-4638-89ec-fa0a6e8a9894 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:21.518037 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad1ebc4-7a5a-4638-89ec-fa0a6e8a9894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:21.572807 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-aad1ebc4-7a5a-4638-89ec-fa0a6e8a9894 None None] MFA Rules not processed for user `db3c1a9e5517403598d1278a14bbab24`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:33:21.576192 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-aad1ebc4-7a5a-4638-89ec-fa0a6e8a9894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:21.687213 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1224/2426] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:33:21 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:33:21.691982 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0652911d-8d36-4e50-bc7b-75a7b8c25c69 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:21.782320 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0652911d-8d36-4e50-bc7b-75a7b8c25c69 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:21.787610 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1fbf72-012d-449d-8873-58e4e13f779c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:21.788309 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1fbf72-012d-449d-8873-58e4e13f779c admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:21.788705 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1fbf72-012d-449d-8873-58e4e13f779c admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:21.788893 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1fbf72-012d-449d-8873-58e4e13f779c admin admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:21.789667 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1fbf72-012d-449d-8873-58e4e13f779c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:21.792798 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1fbf72-012d-449d-8873-58e4e13f779c admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:21.940984 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1203/2427] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:33:21 2022] POST /identity/v3/projects => generated 365 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:33:21.944027 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad1ebc4-7a5a-4638-89ec-fa0a6e8a9894 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:21.958033 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad1ebc4-7a5a-4638-89ec-fa0a6e8a9894 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:22.049504 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c202b504-3410-4dee-bff4-b303464b27a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:22.050275 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c202b504-3410-4dee-bff4-b303464b27a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:22.050557 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c202b504-3410-4dee-bff4-b303464b27a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:22.050836 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c202b504-3410-4dee-bff4-b303464b27a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:22.051593 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c202b504-3410-4dee-bff4-b303464b27a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:22.053787 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c202b504-3410-4dee-bff4-b303464b27a3 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:22.148317 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1225/2428] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:33:21 2022] POST /identity/v3/users => generated 289 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:33:22.153464 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1fbf72-012d-449d-8873-58e4e13f779c admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:22.165353 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1fbf72-012d-449d-8873-58e4e13f779c admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:22.170768 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5bbcda-d161-4314-a56d-50b45d3c7114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:22.171490 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5bbcda-d161-4314-a56d-50b45d3c7114 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:22.171897 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5bbcda-d161-4314-a56d-50b45d3c7114 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:22.172190 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5bbcda-d161-4314-a56d-50b45d3c7114 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:22.173005 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5bbcda-d161-4314-a56d-50b45d3c7114 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:22.174629 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5bbcda-d161-4314-a56d-50b45d3c7114 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:22.187372 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1204/2429] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:33:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:33:22.193255 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c202b504-3410-4dee-bff4-b303464b27a3 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:22.203783 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c202b504-3410-4dee-bff4-b303464b27a3 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:22.208821 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41b5682-a5de-42a1-9d49-fa0cd7d56599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:22.209720 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41b5682-a5de-42a1-9d49-fa0cd7d56599 admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:22.214538 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41b5682-a5de-42a1-9d49-fa0cd7d56599 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:22.214538 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41b5682-a5de-42a1-9d49-fa0cd7d56599 admin admin] PATH_INFO: `/v3/projects/90c06b579f0147ccb9f007e1057c2a51/users/ce2fb76d256447f7a5d39fe021d491de/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:22.224095 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:22.229385 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41b5682-a5de-42a1-9d49-fa0cd7d56599 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90c06b579f0147ccb9f007e1057c2a51, user_id=ce2fb76d256447f7a5d39fe021d491de, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:22.232145 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41b5682-a5de-42a1-9d49-fa0cd7d56599 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:22.318584 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1226/2430] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:33:22 2022] PUT /identity/v3/projects/90c06b579f0147ccb9f007e1057c2a51/users/ce2fb76d256447f7a5d39fe021d491de/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:33:22.323410 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5bbcda-d161-4314-a56d-50b45d3c7114 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:22.334276 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5bbcda-d161-4314-a56d-50b45d3c7114 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:22.403833 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba761587-0980-4404-8a5a-a26abe9683eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:22.404564 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba761587-0980-4404-8a5a-a26abe9683eb admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:22.404847 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba761587-0980-4404-8a5a-a26abe9683eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:22.405203 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba761587-0980-4404-8a5a-a26abe9683eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:22.405952 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba761587-0980-4404-8a5a-a26abe9683eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:22.407652 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba761587-0980-4404-8a5a-a26abe9683eb admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:22.409369 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:22.424942 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1205/2431] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:33:22 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:33:22.429978 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41b5682-a5de-42a1-9d49-fa0cd7d56599 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:22.449158 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41b5682-a5de-42a1-9d49-fa0cd7d56599 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:22.456725 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515508a1-da3c-4426-8eb1-8b3aa4cc2299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:22.457812 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515508a1-da3c-4426-8eb1-8b3aa4cc2299 admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:22.458182 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515508a1-da3c-4426-8eb1-8b3aa4cc2299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:22.458524 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515508a1-da3c-4426-8eb1-8b3aa4cc2299 admin admin] PATH_INFO: `/v3/projects/90c06b579f0147ccb9f007e1057c2a51/users/ce2fb76d256447f7a5d39fe021d491de/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:22.460803 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515508a1-da3c-4426-8eb1-8b3aa4cc2299 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90c06b579f0147ccb9f007e1057c2a51, user_id=ce2fb76d256447f7a5d39fe021d491de, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:22.463576 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515508a1-da3c-4426-8eb1-8b3aa4cc2299 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:22.482146 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1227/2432] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:33:22 2022] PUT /identity/v3/projects/90c06b579f0147ccb9f007e1057c2a51/users/ce2fb76d256447f7a5d39fe021d491de/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:33:22.501296 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d83a87-f70e-4dde-b7e7-755810c6a127 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:22.501799 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d83a87-f70e-4dde-b7e7-755810c6a127 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:22.502297 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d83a87-f70e-4dde-b7e7-755810c6a127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:22.551541 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-50d83a87-f70e-4dde-b7e7-755810c6a127 None None] MFA Rules not processed for user `ce2fb76d256447f7a5d39fe021d491de`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:33:22.552929 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-50d83a87-f70e-4dde-b7e7-755810c6a127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:22.641820 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1206/2433] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:33:22 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:33:22.658074 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515508a1-da3c-4426-8eb1-8b3aa4cc2299 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:22.659999 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-515508a1-da3c-4426-8eb1-8b3aa4cc2299 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:22.733403 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515508a1-da3c-4426-8eb1-8b3aa4cc2299 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:22.738999 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84079132-6f36-4b0d-bffd-afdae532a9b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:22.739708 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84079132-6f36-4b0d-bffd-afdae532a9b3 service cinder] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:22.739977 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84079132-6f36-4b0d-bffd-afdae532a9b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:22.740249 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84079132-6f36-4b0d-bffd-afdae532a9b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:22.754202 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84079132-6f36-4b0d-bffd-afdae532a9b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:22.756090 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84079132-6f36-4b0d-bffd-afdae532a9b3 service cinder] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:22.769586 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1228/2434] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:33:22 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:33:23.049580 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d83a87-f70e-4dde-b7e7-755810c6a127 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:23.061016 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d83a87-f70e-4dde-b7e7-755810c6a127 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:23.067134 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992cb7a8-95b5-4407-ba6d-a8dc2f04e686 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:23.067914 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992cb7a8-95b5-4407-ba6d-a8dc2f04e686 service cinder] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:23.068200 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992cb7a8-95b5-4407-ba6d-a8dc2f04e686 service cinder] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:23.068507 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992cb7a8-95b5-4407-ba6d-a8dc2f04e686 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:23.139492 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992cb7a8-95b5-4407-ba6d-a8dc2f04e686 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:23.141022 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992cb7a8-95b5-4407-ba6d-a8dc2f04e686 service cinder] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:23.153106 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1207/2435] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:33:23 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:33:25.500489 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19acf090-8a28-4668-9154-4a9b6d14c3b3 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:25.501932 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19acf090-8a28-4668-9154-4a9b6d14c3b3 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:25.502447 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19acf090-8a28-4668-9154-4a9b6d14c3b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:25.530211 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-19acf090-8a28-4668-9154-4a9b6d14c3b3 None None] MFA Rules not processed for user `fcfcd9f134e74b09bfdac337c81c8a07`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:33:25.531871 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-19acf090-8a28-4668-9154-4a9b6d14c3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:25.601136 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1229/2436] 10.222.0.23 () {58 vars in 967 bytes} [Tue Jun 21 18:33:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:33:25.661447 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992cb7a8-95b5-4407-ba6d-a8dc2f04e686 service cinder] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:25.663002 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-992cb7a8-95b5-4407-ba6d-a8dc2f04e686 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:25.769104 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992cb7a8-95b5-4407-ba6d-a8dc2f04e686 service cinder] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:25.774421 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed67ae9-cf27-47e0-ac28-aa8537c2f770 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e0d25565f1543d9a7577288fc09119f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:25.775907 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed67ae9-cf27-47e0-ac28-aa8537c2f770 service swift] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:25.776335 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed67ae9-cf27-47e0-ac28-aa8537c2f770 service swift] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:25.776650 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed67ae9-cf27-47e0-ac28-aa8537c2f770 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:25.786709 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed67ae9-cf27-47e0-ac28-aa8537c2f770 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:25.788483 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed67ae9-cf27-47e0-ac28-aa8537c2f770 service swift] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:25.797708 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1208/2437] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Jun 21 18:33:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 18:33:27.104699 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19acf090-8a28-4668-9154-4a9b6d14c3b3 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:27.202418 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19acf090-8a28-4668-9154-4a9b6d14c3b3 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:27.207180 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b79146-2b30-4175-b3a1-0fb2de92051b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:27.208281 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b79146-2b30-4175-b3a1-0fb2de92051b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:27.208598 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b79146-2b30-4175-b3a1-0fb2de92051b admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:27.208879 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b79146-2b30-4175-b3a1-0fb2de92051b admin admin] PATH_INFO: `/v3/users/db3c1a9e5517403598d1278a14bbab24` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:27.210237 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b79146-2b30-4175-b3a1-0fb2de92051b admin admin] RBAC: Authorizing `identity:delete_user(user_id=db3c1a9e5517403598d1278a14bbab24)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:27.212982 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b79146-2b30-4175-b3a1-0fb2de92051b admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:27.285901 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:27.404872 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-a6b79146-2b30-4175-b3a1-0fb2de92051b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db3c1a9e5517403598d1278a14bbab24'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:27.419826 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-a6b79146-2b30-4175-b3a1-0fb2de92051b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db3c1a9e5517403598d1278a14bbab24'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:27.480809 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-a6b79146-2b30-4175-b3a1-0fb2de92051b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db3c1a9e5517403598d1278a14bbab24'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:27.481632 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-a6b79146-2b30-4175-b3a1-0fb2de92051b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db3c1a9e5517403598d1278a14bbab24'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:27.491137 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-a6b79146-2b30-4175-b3a1-0fb2de92051b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db3c1a9e5517403598d1278a14bbab24'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:27.504311 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1230/2438] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:33:27 2022] DELETE /identity/v3/users/db3c1a9e5517403598d1278a14bbab24 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:33:27.513020 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393955e5-146c-4a30-ad44-f275c2d4115a None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:27.513817 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393955e5-146c-4a30-ad44-f275c2d4115a None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:27.514275 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393955e5-146c-4a30-ad44-f275c2d4115a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:27.543448 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-393955e5-146c-4a30-ad44-f275c2d4115a None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:33:27.544728 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-393955e5-146c-4a30-ad44-f275c2d4115a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:27.549291 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:27.620433 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1209/2439] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:33:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:33:27.628828 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b79146-2b30-4175-b3a1-0fb2de92051b admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:27.630391 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-a6b79146-2b30-4175-b3a1-0fb2de92051b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:27.714976 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b79146-2b30-4175-b3a1-0fb2de92051b admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:27.719579 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7996de-455b-4454-8dff-bb87059d5c82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:27.720294 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7996de-455b-4454-8dff-bb87059d5c82 service neutron] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:27.720615 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7996de-455b-4454-8dff-bb87059d5c82 service neutron] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:27.720896 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7996de-455b-4454-8dff-bb87059d5c82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:27.733043 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7996de-455b-4454-8dff-bb87059d5c82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:27.734559 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7996de-455b-4454-8dff-bb87059d5c82 service neutron] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:27.747749 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1231/2440] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:33:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:33:28.752779 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393955e5-146c-4a30-ad44-f275c2d4115a None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:28.754780 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-393955e5-146c-4a30-ad44-f275c2d4115a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:28.770771 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393955e5-146c-4a30-ad44-f275c2d4115a None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:28.776121 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7a256c-aceb-453b-b45b-09ef9c3adcc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:28.779282 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a256c-aceb-453b-b45b-09ef9c3adcc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:28.779946 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a256c-aceb-453b-b45b-09ef9c3adcc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:28.780195 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a256c-aceb-453b-b45b-09ef9c3adcc7 admin admin] PATH_INFO: `/v3/projects/92ea2d080d0849e29ee7db503c1ee07d` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:28.782506 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7a256c-aceb-453b-b45b-09ef9c3adcc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92ea2d080d0849e29ee7db503c1ee07d)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:28.783559 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7a256c-aceb-453b-b45b-09ef9c3adcc7 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:28.929205 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-fc7a256c-aceb-453b-b45b-09ef9c3adcc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92ea2d080d0849e29ee7db503c1ee07d', 'request_id': 'req-fc7a256c-aceb-453b-b45b-09ef9c3adcc7'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:29.046160 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-fc7a256c-aceb-453b-b45b-09ef9c3adcc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92ea2d080d0849e29ee7db503c1ee07d', 'request_id': 'req-fc7a256c-aceb-453b-b45b-09ef9c3adcc7'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:29.055681 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-fc7a256c-aceb-453b-b45b-09ef9c3adcc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92ea2d080d0849e29ee7db503c1ee07d', 'request_id': 'req-fc7a256c-aceb-453b-b45b-09ef9c3adcc7'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:29.075329 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-fc7a256c-aceb-453b-b45b-09ef9c3adcc7 admin admin] The token cache is being invalidate because project 92ea2d080d0849e29ee7db503c1ee07d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:33:29.075753 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-fc7a256c-aceb-453b-b45b-09ef9c3adcc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:29.076743 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-fc7a256c-aceb-453b-b45b-09ef9c3adcc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:29.078474 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1210/2441] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:33:28 2022] DELETE /identity/v3/projects/92ea2d080d0849e29ee7db503c1ee07d => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:33:29.082768 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7996de-455b-4454-8dff-bb87059d5c82 service neutron] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:29.093623 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7996de-455b-4454-8dff-bb87059d5c82 service neutron] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:29.243845 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea27929-7997-45c5-8ffa-4dc55873371d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:29.248051 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea27929-7997-45c5-8ffa-4dc55873371d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:29.249127 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea27929-7997-45c5-8ffa-4dc55873371d admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:29.250059 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea27929-7997-45c5-8ffa-4dc55873371d admin admin] PATH_INFO: `/v3/users/ce2fb76d256447f7a5d39fe021d491de` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:29.254106 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea27929-7997-45c5-8ffa-4dc55873371d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce2fb76d256447f7a5d39fe021d491de)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:29.260474 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea27929-7997-45c5-8ffa-4dc55873371d admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:29.477448 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-0ea27929-7997-45c5-8ffa-4dc55873371d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce2fb76d256447f7a5d39fe021d491de'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:29.480040 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:29.497971 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-0ea27929-7997-45c5-8ffa-4dc55873371d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce2fb76d256447f7a5d39fe021d491de'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:29.577088 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-0ea27929-7997-45c5-8ffa-4dc55873371d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce2fb76d256447f7a5d39fe021d491de'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:29.577804 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-0ea27929-7997-45c5-8ffa-4dc55873371d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce2fb76d256447f7a5d39fe021d491de'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:29.591742 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-0ea27929-7997-45c5-8ffa-4dc55873371d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce2fb76d256447f7a5d39fe021d491de'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:29.600847 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1232/2442] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:33:29 2022] DELETE /identity/v3/users/ce2fb76d256447f7a5d39fe021d491de => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:33:30.532018 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7a256c-aceb-453b-b45b-09ef9c3adcc7 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:30.535520 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-fc7a256c-aceb-453b-b45b-09ef9c3adcc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:30.584719 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7a256c-aceb-453b-b45b-09ef9c3adcc7 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:30.679736 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea27929-7997-45c5-8ffa-4dc55873371d admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:30.687274 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-0ea27929-7997-45c5-8ffa-4dc55873371d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:30.747230 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3651b16-4049-44ae-8142-0a5af39ccd33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:30.748952 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3651b16-4049-44ae-8142-0a5af39ccd33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:30.750468 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3651b16-4049-44ae-8142-0a5af39ccd33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:30.751084 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3651b16-4049-44ae-8142-0a5af39ccd33 admin admin] PATH_INFO: `/v3/projects/90c06b579f0147ccb9f007e1057c2a51` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:30.753998 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3651b16-4049-44ae-8142-0a5af39ccd33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90c06b579f0147ccb9f007e1057c2a51)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:30.758077 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3651b16-4049-44ae-8142-0a5af39ccd33 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:30.839267 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea27929-7997-45c5-8ffa-4dc55873371d admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:30.842234 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdee6c3-fd10-427e-a20f-8852b6d0e42e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904255dd3c1f406eb10f6fc65f9cd098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:30.843030 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdee6c3-fd10-427e-a20f-8852b6d0e42e service nova] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:30.843655 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdee6c3-fd10-427e-a20f-8852b6d0e42e service nova] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:30.844009 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdee6c3-fd10-427e-a20f-8852b6d0e42e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:30.848349 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-9fdee6c3-fd10-427e-a20f-8852b6d0e42e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:30.869108 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdee6c3-fd10-427e-a20f-8852b6d0e42e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:30.878060 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdee6c3-fd10-427e-a20f-8852b6d0e42e service nova] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:30.886299 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1233/2443] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:33:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:33:30.921981 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:30.991675 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-c3651b16-4049-44ae-8142-0a5af39ccd33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90c06b579f0147ccb9f007e1057c2a51', 'request_id': 'req-c3651b16-4049-44ae-8142-0a5af39ccd33'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:31.091647 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-c3651b16-4049-44ae-8142-0a5af39ccd33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90c06b579f0147ccb9f007e1057c2a51', 'request_id': 'req-c3651b16-4049-44ae-8142-0a5af39ccd33'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:31.105526 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-c3651b16-4049-44ae-8142-0a5af39ccd33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90c06b579f0147ccb9f007e1057c2a51', 'request_id': 'req-c3651b16-4049-44ae-8142-0a5af39ccd33'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:31.127518 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-c3651b16-4049-44ae-8142-0a5af39ccd33 admin admin] The token cache is being invalidate because project 90c06b579f0147ccb9f007e1057c2a51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:33:31.128241 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-c3651b16-4049-44ae-8142-0a5af39ccd33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:31.129792 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-c3651b16-4049-44ae-8142-0a5af39ccd33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:31.132460 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1211/2444] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:33:30 2022] DELETE /identity/v3/projects/90c06b579f0147ccb9f007e1057c2a51 => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:33:31.141137 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8130f4f8-56ca-41f4-a898-8b501a1e323d None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:31.141137 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8130f4f8-56ca-41f4-a898-8b501a1e323d None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:31.141137 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8130f4f8-56ca-41f4-a898-8b501a1e323d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:31.164530 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-8130f4f8-56ca-41f4-a898-8b501a1e323d None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:33:31.166557 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-8130f4f8-56ca-41f4-a898-8b501a1e323d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:31.249835 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1234/2445] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:33:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:33:31.262913 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230487f-71f9-481f-ba3a-c83233e6b032 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:31.267330 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230487f-71f9-481f-ba3a-c83233e6b032 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:31.267330 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230487f-71f9-481f-ba3a-c83233e6b032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:31.290333 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-e230487f-71f9-481f-ba3a-c83233e6b032 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:33:31.292804 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-e230487f-71f9-481f-ba3a-c83233e6b032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:31.301299 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1212/2446] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:33:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:33:31.307679 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8130f4f8-56ca-41f4-a898-8b501a1e323d None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:31.327736 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8130f4f8-56ca-41f4-a898-8b501a1e323d None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:31.334532 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01d0b20-ac62-4d21-acb0-8a28fcbaf4db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:31.335485 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d0b20-ac62-4d21-acb0-8a28fcbaf4db admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:31.336029 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d0b20-ac62-4d21-acb0-8a28fcbaf4db admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:31.336577 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d0b20-ac62-4d21-acb0-8a28fcbaf4db admin admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:31.338615 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01d0b20-ac62-4d21-acb0-8a28fcbaf4db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:33:31.339243 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01d0b20-ac62-4d21-acb0-8a28fcbaf4db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:31.345007 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01d0b20-ac62-4d21-acb0-8a28fcbaf4db admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:31.378138 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1235/2447] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:33:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:33:31.384936 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e230487f-71f9-481f-ba3a-c83233e6b032 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:31.407523 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e230487f-71f9-481f-ba3a-c83233e6b032 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:31.420886 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f37f57-2acd-4fe4-b92c-4984e007ba5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:31.422197 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f37f57-2acd-4fe4-b92c-4984e007ba5e admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:31.428440 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f37f57-2acd-4fe4-b92c-4984e007ba5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:31.429027 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f37f57-2acd-4fe4-b92c-4984e007ba5e admin admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:31.430511 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f37f57-2acd-4fe4-b92c-4984e007ba5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:31.433670 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f37f57-2acd-4fe4-b92c-4984e007ba5e admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:31.589096 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1213/2448] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:33:31 2022] POST /identity/v3/projects => generated 389 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:33:31.595379 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01d0b20-ac62-4d21-acb0-8a28fcbaf4db admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:31.648854 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01d0b20-ac62-4d21-acb0-8a28fcbaf4db admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:31.728126 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:31.813641 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a3b117-f0aa-440c-9674-6e7f928fa394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:31.813641 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a3b117-f0aa-440c-9674-6e7f928fa394 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:31.813641 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a3b117-f0aa-440c-9674-6e7f928fa394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:31.813641 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a3b117-f0aa-440c-9674-6e7f928fa394 admin admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:31.815718 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a3b117-f0aa-440c-9674-6e7f928fa394 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:31.815718 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a3b117-f0aa-440c-9674-6e7f928fa394 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:31.939800 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1236/2449] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:33:31 2022] POST /identity/v3/users => generated 295 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:33:31.945684 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f37f57-2acd-4fe4-b92c-4984e007ba5e admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:31.972187 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f37f57-2acd-4fe4-b92c-4984e007ba5e admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:31.985440 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740e8a30-8311-45d5-ac66-5e1a63969d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:31.986140 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e8a30-8311-45d5-ac66-5e1a63969d2d admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:31.986423 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e8a30-8311-45d5-ac66-5e1a63969d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:31.986714 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e8a30-8311-45d5-ac66-5e1a63969d2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:31.987451 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740e8a30-8311-45d5-ac66-5e1a63969d2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:31.989116 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740e8a30-8311-45d5-ac66-5e1a63969d2d admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:32.022002 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1214/2450] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:33:31 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:33:32.026350 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a3b117-f0aa-440c-9674-6e7f928fa394 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:32.040262 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a3b117-f0aa-440c-9674-6e7f928fa394 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:32.046804 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f094f2-f0a5-4149-87f2-0cf2c526bde8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:32.047657 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f094f2-f0a5-4149-87f2-0cf2c526bde8 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:32.048044 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f094f2-f0a5-4149-87f2-0cf2c526bde8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:32.048451 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f094f2-f0a5-4149-87f2-0cf2c526bde8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:32.049501 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f094f2-f0a5-4149-87f2-0cf2c526bde8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:32.051318 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f094f2-f0a5-4149-87f2-0cf2c526bde8 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:32.066914 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1237/2451] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:33:32 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:33:32.073982 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740e8a30-8311-45d5-ac66-5e1a63969d2d admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:32.091700 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740e8a30-8311-45d5-ac66-5e1a63969d2d admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:32.100969 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f804917f-1e91-46bc-a54c-d1652ca8ec77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:32.103009 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f804917f-1e91-46bc-a54c-d1652ca8ec77 admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:32.103683 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f804917f-1e91-46bc-a54c-d1652ca8ec77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:32.104299 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f804917f-1e91-46bc-a54c-d1652ca8ec77 admin admin] PATH_INFO: `/v3/projects/7de778ebf93f4ecbb8fd0f9d0eca9b9a/users/be250c8f6aa74120a17f64f980b318a5/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:32.135313 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f804917f-1e91-46bc-a54c-d1652ca8ec77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7de778ebf93f4ecbb8fd0f9d0eca9b9a, user_id=be250c8f6aa74120a17f64f980b318a5, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:32.142155 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f804917f-1e91-46bc-a54c-d1652ca8ec77 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:32.261691 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1215/2452] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:33:32 2022] PUT /identity/v3/projects/7de778ebf93f4ecbb8fd0f9d0eca9b9a/users/be250c8f6aa74120a17f64f980b318a5/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:33:32.272989 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78ddd2d-4ad4-431f-86e2-084a4d71da49 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:32.273459 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78ddd2d-4ad4-431f-86e2-084a4d71da49 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:32.273921 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78ddd2d-4ad4-431f-86e2-084a4d71da49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:32.308919 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-b78ddd2d-4ad4-431f-86e2-084a4d71da49 None None] MFA Rules not processed for user `be250c8f6aa74120a17f64f980b318a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:33:32.310191 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-b78ddd2d-4ad4-431f-86e2-084a4d71da49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:32.444817 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1238/2453] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:33:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:33:32.449514 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f804917f-1e91-46bc-a54c-d1652ca8ec77 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:32.562240 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695ab816-30ad-4f21-8f27-1d79e8f1ce13 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:32.563222 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695ab816-30ad-4f21-8f27-1d79e8f1ce13 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:32.564083 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695ab816-30ad-4f21-8f27-1d79e8f1ce13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:32.604035 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-695ab816-30ad-4f21-8f27-1d79e8f1ce13 None None] MFA Rules not processed for user `fcfcd9f134e74b09bfdac337c81c8a07`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:33:32.606593 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-695ab816-30ad-4f21-8f27-1d79e8f1ce13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:32.644153 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f804917f-1e91-46bc-a54c-d1652ca8ec77 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:32.658889 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0899ea0c-e0bf-4b65-afbb-820b10e0b2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:32.660146 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0899ea0c-e0bf-4b65-afbb-820b10e0b2a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:32.660760 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0899ea0c-e0bf-4b65-afbb-820b10e0b2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:32.674447 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0899ea0c-e0bf-4b65-afbb-820b10e0b2a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:32.677210 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0899ea0c-e0bf-4b65-afbb-820b10e0b2a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:32.694548 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0899ea0c-e0bf-4b65-afbb-820b10e0b2a5 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:32.754108 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1239/2454] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Jun 21 18:33:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:33:32.773102 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695ab816-30ad-4f21-8f27-1d79e8f1ce13 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:32.775314 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-695ab816-30ad-4f21-8f27-1d79e8f1ce13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:32.823445 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1216/2455] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:33:32 2022] POST /identity/v3/projects => generated 389 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:33:32.832275 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0899ea0c-e0bf-4b65-afbb-820b10e0b2a5 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:32.908150 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:32.934893 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:32.940300 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695ab816-30ad-4f21-8f27-1d79e8f1ce13 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:32.959270 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0899ea0c-e0bf-4b65-afbb-820b10e0b2a5 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:32.967466 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283efc49-9233-47be-bef7-10e0028d3a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:32.969917 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283efc49-9233-47be-bef7-10e0028d3a2b admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:32.970475 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283efc49-9233-47be-bef7-10e0028d3a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:32.971070 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283efc49-9233-47be-bef7-10e0028d3a2b admin admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:32.974332 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283efc49-9233-47be-bef7-10e0028d3a2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:32.979586 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283efc49-9233-47be-bef7-10e0028d3a2b admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:33.043751 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7195e784-1818-4118-8eaf-285924e69e9d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e0d25565f1543d9a7577288fc09119f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:33.045559 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7195e784-1818-4118-8eaf-285924e69e9d service swift] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:33.046050 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7195e784-1818-4118-8eaf-285924e69e9d service swift] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:33.046432 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7195e784-1818-4118-8eaf-285924e69e9d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:33.145383 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7195e784-1818-4118-8eaf-285924e69e9d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:33.148433 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7195e784-1818-4118-8eaf-285924e69e9d service swift] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:33.182946 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1240/2456] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Jun 21 18:33:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 21 18:33:33.225374 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1217/2457] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:33:32 2022] POST /identity/v3/users => generated 301 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:33:33.229891 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7195e784-1818-4118-8eaf-285924e69e9d service swift] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:33.243206 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7195e784-1818-4118-8eaf-285924e69e9d service swift] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:33.249984 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406c8468-94f3-49fe-859c-fe574b018819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:33.253758 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c8468-94f3-49fe-859c-fe574b018819 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:33.253758 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c8468-94f3-49fe-859c-fe574b018819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:33.253758 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c8468-94f3-49fe-859c-fe574b018819 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:33.254755 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406c8468-94f3-49fe-859c-fe574b018819 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:33.258238 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406c8468-94f3-49fe-859c-fe574b018819 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:33.295057 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283efc49-9233-47be-bef7-10e0028d3a2b admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:33.310391 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1241/2458] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:33:33 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:33:33.328464 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283efc49-9233-47be-bef7-10e0028d3a2b admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:33.339739 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feab150-1e70-4cdd-97ac-51f889076a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:33.341438 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feab150-1e70-4cdd-97ac-51f889076a51 admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:33.342308 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feab150-1e70-4cdd-97ac-51f889076a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:33.343062 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feab150-1e70-4cdd-97ac-51f889076a51 admin admin] PATH_INFO: `/v3/projects/2a9c539e6fbe4698a0990926adb232c0/users/61b9e73198e449df8ad22b0ba25a04ed/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:33.390359 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feab150-1e70-4cdd-97ac-51f889076a51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a9c539e6fbe4698a0990926adb232c0, user_id=61b9e73198e449df8ad22b0ba25a04ed, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:33.396135 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feab150-1e70-4cdd-97ac-51f889076a51 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:33.533653 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406c8468-94f3-49fe-859c-fe574b018819 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:33.534503 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1218/2459] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:33:33 2022] PUT /identity/v3/projects/2a9c539e6fbe4698a0990926adb232c0/users/61b9e73198e449df8ad22b0ba25a04ed/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:33:33.554319 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406c8468-94f3-49fe-859c-fe574b018819 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:33.706261 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c460f14-9430-4713-ad3c-05ed4e117d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:33.707503 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c460f14-9430-4713-ad3c-05ed4e117d99 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:33.707998 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c460f14-9430-4713-ad3c-05ed4e117d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:33.708776 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c460f14-9430-4713-ad3c-05ed4e117d99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:33.711110 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c460f14-9430-4713-ad3c-05ed4e117d99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:33.715324 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c460f14-9430-4713-ad3c-05ed4e117d99 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:33.738538 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1242/2460] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:33:33 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:33:33.744182 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feab150-1e70-4cdd-97ac-51f889076a51 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:33.767983 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feab150-1e70-4cdd-97ac-51f889076a51 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:33.780282 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b08a59-5e44-46cc-af4c-d05bbddf920f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:33.782081 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b08a59-5e44-46cc-af4c-d05bbddf920f admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:33.782522 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b08a59-5e44-46cc-af4c-d05bbddf920f admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:33.782950 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b08a59-5e44-46cc-af4c-d05bbddf920f admin admin] PATH_INFO: `/v3/projects/2a9c539e6fbe4698a0990926adb232c0/users/61b9e73198e449df8ad22b0ba25a04ed/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:33.787331 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b08a59-5e44-46cc-af4c-d05bbddf920f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a9c539e6fbe4698a0990926adb232c0, user_id=61b9e73198e449df8ad22b0ba25a04ed, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:33.792984 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b08a59-5e44-46cc-af4c-d05bbddf920f admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:33.815938 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1219/2461] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:33:33 2022] PUT /identity/v3/projects/2a9c539e6fbe4698a0990926adb232c0/users/61b9e73198e449df8ad22b0ba25a04ed/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:33:33.842611 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca46a7-dc9e-4c23-8ea2-dc879cb64110 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:33.843452 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca46a7-dc9e-4c23-8ea2-dc879cb64110 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:33.843452 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca46a7-dc9e-4c23-8ea2-dc879cb64110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:33.907047 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-f6ca46a7-dc9e-4c23-8ea2-dc879cb64110 None None] MFA Rules not processed for user `61b9e73198e449df8ad22b0ba25a04ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:33:33.907047 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-f6ca46a7-dc9e-4c23-8ea2-dc879cb64110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:33.916739 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:34.026596 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1243/2462] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:33:33 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:33:34.042756 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b08a59-5e44-46cc-af4c-d05bbddf920f admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:34.044654 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-43b08a59-5e44-46cc-af4c-d05bbddf920f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:34.169283 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b08a59-5e44-46cc-af4c-d05bbddf920f admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:34.173705 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e924e576-2b42-4341-9601-5a6e84985d6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:34.174502 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e924e576-2b42-4341-9601-5a6e84985d6c service cinder] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:34.175029 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e924e576-2b42-4341-9601-5a6e84985d6c service cinder] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:34.175439 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e924e576-2b42-4341-9601-5a6e84985d6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:34.187452 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e924e576-2b42-4341-9601-5a6e84985d6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:34.189077 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e924e576-2b42-4341-9601-5a6e84985d6c service cinder] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:34.201019 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1220/2463] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:33:34 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:33:34.378607 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ca46a7-dc9e-4c23-8ea2-dc879cb64110 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:34.526592 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ca46a7-dc9e-4c23-8ea2-dc879cb64110 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:34.533245 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef13bb87-3b4c-4918-b10e-9c382e70dab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:34.534158 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef13bb87-3b4c-4918-b10e-9c382e70dab4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:34.534450 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef13bb87-3b4c-4918-b10e-9c382e70dab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:34.534709 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef13bb87-3b4c-4918-b10e-9c382e70dab4 admin admin] PATH_INFO: `/v3/users/be250c8f6aa74120a17f64f980b318a5` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:34.536943 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef13bb87-3b4c-4918-b10e-9c382e70dab4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be250c8f6aa74120a17f64f980b318a5)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:34.542045 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef13bb87-3b4c-4918-b10e-9c382e70dab4 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:34.818072 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-ef13bb87-3b4c-4918-b10e-9c382e70dab4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be250c8f6aa74120a17f64f980b318a5'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:34.836591 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-ef13bb87-3b4c-4918-b10e-9c382e70dab4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be250c8f6aa74120a17f64f980b318a5'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:34.942181 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-ef13bb87-3b4c-4918-b10e-9c382e70dab4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be250c8f6aa74120a17f64f980b318a5'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:34.943823 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-ef13bb87-3b4c-4918-b10e-9c382e70dab4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be250c8f6aa74120a17f64f980b318a5'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:34.961390 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-ef13bb87-3b4c-4918-b10e-9c382e70dab4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be250c8f6aa74120a17f64f980b318a5'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:34.975885 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1244/2464] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:33:34 2022] DELETE /identity/v3/users/be250c8f6aa74120a17f64f980b318a5 => generated 0 bytes in 598 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:33:34.980640 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b5f6e8-46f0-4a72-8142-6c47e770c54f None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:34.981255 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b5f6e8-46f0-4a72-8142-6c47e770c54f None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:34.981378 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b5f6e8-46f0-4a72-8142-6c47e770c54f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:35.004667 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-61b5f6e8-46f0-4a72-8142-6c47e770c54f None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:33:35.006251 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-61b5f6e8-46f0-4a72-8142-6c47e770c54f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:35.150678 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1221/2465] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:33:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:33:35.163083 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef13bb87-3b4c-4918-b10e-9c382e70dab4 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:35.164761 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-ef13bb87-3b4c-4918-b10e-9c382e70dab4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:35.215351 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:35.303363 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef13bb87-3b4c-4918-b10e-9c382e70dab4 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:35.312487 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0804afa4-9f66-4471-abfa-0aba7047cda7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:35.312487 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0804afa4-9f66-4471-abfa-0aba7047cda7 service neutron] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:35.312487 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0804afa4-9f66-4471-abfa-0aba7047cda7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:35.312487 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0804afa4-9f66-4471-abfa-0aba7047cda7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:35.328663 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0804afa4-9f66-4471-abfa-0aba7047cda7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:35.330992 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0804afa4-9f66-4471-abfa-0aba7047cda7 service neutron] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:35.344432 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1245/2466] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:33:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:33:36.309610 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b5f6e8-46f0-4a72-8142-6c47e770c54f None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:36.311275 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-61b5f6e8-46f0-4a72-8142-6c47e770c54f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:36.357204 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b5f6e8-46f0-4a72-8142-6c47e770c54f None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:36.364828 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e2d463-bb93-45a0-bd87-bff3e97401fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:36.366003 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e2d463-bb93-45a0-bd87-bff3e97401fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:36.366462 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e2d463-bb93-45a0-bd87-bff3e97401fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:36.366905 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e2d463-bb93-45a0-bd87-bff3e97401fc admin admin] PATH_INFO: `/v3/projects/7de778ebf93f4ecbb8fd0f9d0eca9b9a` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:36.369194 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e2d463-bb93-45a0-bd87-bff3e97401fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=7de778ebf93f4ecbb8fd0f9d0eca9b9a)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:36.371388 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e2d463-bb93-45a0-bd87-bff3e97401fc admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:36.537538 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:36.564103 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-42e2d463-bb93-45a0-bd87-bff3e97401fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7de778ebf93f4ecbb8fd0f9d0eca9b9a', 'request_id': 'req-42e2d463-bb93-45a0-bd87-bff3e97401fc'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:36.661964 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-42e2d463-bb93-45a0-bd87-bff3e97401fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7de778ebf93f4ecbb8fd0f9d0eca9b9a', 'request_id': 'req-42e2d463-bb93-45a0-bd87-bff3e97401fc'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:36.673945 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-42e2d463-bb93-45a0-bd87-bff3e97401fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7de778ebf93f4ecbb8fd0f9d0eca9b9a', 'request_id': 'req-42e2d463-bb93-45a0-bd87-bff3e97401fc'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:36.708022 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-42e2d463-bb93-45a0-bd87-bff3e97401fc admin admin] The token cache is being invalidate because project 7de778ebf93f4ecbb8fd0f9d0eca9b9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:33:36.709153 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-42e2d463-bb93-45a0-bd87-bff3e97401fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:36.715130 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-42e2d463-bb93-45a0-bd87-bff3e97401fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:36.717616 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1222/2467] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:33:36 2022] DELETE /identity/v3/projects/7de778ebf93f4ecbb8fd0f9d0eca9b9a => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:33:36.721749 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0804afa4-9f66-4471-abfa-0aba7047cda7 service neutron] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:36.743467 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0804afa4-9f66-4471-abfa-0aba7047cda7 service neutron] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:36.884441 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f7ba6e-e14b-47b7-8ebe-98f29b662613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:36.884441 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7ba6e-e14b-47b7-8ebe-98f29b662613 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:36.884441 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7ba6e-e14b-47b7-8ebe-98f29b662613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:36.884441 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7ba6e-e14b-47b7-8ebe-98f29b662613 admin admin] PATH_INFO: `/v3/users/61b9e73198e449df8ad22b0ba25a04ed` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:36.886614 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f7ba6e-e14b-47b7-8ebe-98f29b662613 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61b9e73198e449df8ad22b0ba25a04ed)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:36.888451 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f7ba6e-e14b-47b7-8ebe-98f29b662613 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:37.165642 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-66f7ba6e-e14b-47b7-8ebe-98f29b662613 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61b9e73198e449df8ad22b0ba25a04ed'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:37.211082 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-66f7ba6e-e14b-47b7-8ebe-98f29b662613 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61b9e73198e449df8ad22b0ba25a04ed'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:37.322183 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-66f7ba6e-e14b-47b7-8ebe-98f29b662613 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61b9e73198e449df8ad22b0ba25a04ed'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:37.323364 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-66f7ba6e-e14b-47b7-8ebe-98f29b662613 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61b9e73198e449df8ad22b0ba25a04ed'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:37.357220 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-66f7ba6e-e14b-47b7-8ebe-98f29b662613 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61b9e73198e449df8ad22b0ba25a04ed'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:37.378773 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1246/2468] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:33:36 2022] DELETE /identity/v3/users/61b9e73198e449df8ad22b0ba25a04ed => generated 0 bytes in 657 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:33:38.740701 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e2d463-bb93-45a0-bd87-bff3e97401fc admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:38.744472 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-42e2d463-bb93-45a0-bd87-bff3e97401fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:38.810621 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e2d463-bb93-45a0-bd87-bff3e97401fc admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:38.989251 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5461fc-ea3a-44f0-8f01-040922eab160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:38.990228 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5461fc-ea3a-44f0-8f01-040922eab160 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:38.990518 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5461fc-ea3a-44f0-8f01-040922eab160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:38.990796 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5461fc-ea3a-44f0-8f01-040922eab160 admin admin] PATH_INFO: `/v3/projects/2a9c539e6fbe4698a0990926adb232c0` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:38.992524 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5461fc-ea3a-44f0-8f01-040922eab160 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a9c539e6fbe4698a0990926adb232c0)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:38.996174 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5461fc-ea3a-44f0-8f01-040922eab160 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:39.139069 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-5b5461fc-ea3a-44f0-8f01-040922eab160 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a9c539e6fbe4698a0990926adb232c0', 'request_id': 'req-5b5461fc-ea3a-44f0-8f01-040922eab160'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:39.220490 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-5b5461fc-ea3a-44f0-8f01-040922eab160 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a9c539e6fbe4698a0990926adb232c0', 'request_id': 'req-5b5461fc-ea3a-44f0-8f01-040922eab160'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:39.266240 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-5b5461fc-ea3a-44f0-8f01-040922eab160 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a9c539e6fbe4698a0990926adb232c0', 'request_id': 'req-5b5461fc-ea3a-44f0-8f01-040922eab160'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:39.322819 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-5b5461fc-ea3a-44f0-8f01-040922eab160 admin admin] The token cache is being invalidate because project 2a9c539e6fbe4698a0990926adb232c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:33:39.323253 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-5b5461fc-ea3a-44f0-8f01-040922eab160 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:39.331240 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-5b5461fc-ea3a-44f0-8f01-040922eab160 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:39.337782 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1223/2469] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:33:38 2022] DELETE /identity/v3/projects/2a9c539e6fbe4698a0990926adb232c0 => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:33:39.339678 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45dd9eb-3e58-4b10-9c15-b0aace1b22a1 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:39.340615 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45dd9eb-3e58-4b10-9c15-b0aace1b22a1 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:39.341149 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45dd9eb-3e58-4b10-9c15-b0aace1b22a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:39.373260 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-a45dd9eb-3e58-4b10-9c15-b0aace1b22a1 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:33:39.375453 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-a45dd9eb-3e58-4b10-9c15-b0aace1b22a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:39.434996 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:39.465869 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf48f6-1426-4ce8-9d10-109f5ab7d4f9 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:39.466416 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf48f6-1426-4ce8-9d10-109f5ab7d4f9 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:39.466734 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf48f6-1426-4ce8-9d10-109f5ab7d4f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:39.474681 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1247/2470] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:33:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:33:39.488488 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-92bf48f6-1426-4ce8-9d10-109f5ab7d4f9 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:33:39.490135 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-92bf48f6-1426-4ce8-9d10-109f5ab7d4f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:39.501574 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1224/2471] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:33:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:33:39.504143 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45dd9eb-3e58-4b10-9c15-b0aace1b22a1 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:39.517278 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45dd9eb-3e58-4b10-9c15-b0aace1b22a1 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:39.529047 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95661e63-222f-4efb-8eea-41de8b31dc6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:39.529800 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95661e63-222f-4efb-8eea-41de8b31dc6d admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:39.530088 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95661e63-222f-4efb-8eea-41de8b31dc6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:39.530453 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95661e63-222f-4efb-8eea-41de8b31dc6d admin admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:39.531385 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95661e63-222f-4efb-8eea-41de8b31dc6d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:33:39.531698 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95661e63-222f-4efb-8eea-41de8b31dc6d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:39.533408 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95661e63-222f-4efb-8eea-41de8b31dc6d admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:39.572997 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1248/2472] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:33:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:33:39.577720 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf48f6-1426-4ce8-9d10-109f5ab7d4f9 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:39.589019 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf48f6-1426-4ce8-9d10-109f5ab7d4f9 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:39.594380 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3116322b-ca2c-4ee2-9004-693b2177b929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:39.595121 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3116322b-ca2c-4ee2-9004-693b2177b929 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:39.595366 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3116322b-ca2c-4ee2-9004-693b2177b929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:39.595722 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3116322b-ca2c-4ee2-9004-693b2177b929 admin admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:39.596558 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3116322b-ca2c-4ee2-9004-693b2177b929 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:39.599093 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3116322b-ca2c-4ee2-9004-693b2177b929 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:39.716418 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1225/2473] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:33:39 2022] POST /identity/v3/projects => generated 383 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:33:39.721227 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95661e63-222f-4efb-8eea-41de8b31dc6d admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:39.735249 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95661e63-222f-4efb-8eea-41de8b31dc6d admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:39.853221 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e37a72-25bb-46e4-a10a-9687fc2eeb7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:39.854424 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e37a72-25bb-46e4-a10a-9687fc2eeb7f admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:39.854710 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e37a72-25bb-46e4-a10a-9687fc2eeb7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:39.855117 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e37a72-25bb-46e4-a10a-9687fc2eeb7f admin admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:39.855870 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e37a72-25bb-46e4-a10a-9687fc2eeb7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:39.860413 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e37a72-25bb-46e4-a10a-9687fc2eeb7f admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:39.944108 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3116322b-ca2c-4ee2-9004-693b2177b929 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:39.946524 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-3116322b-ca2c-4ee2-9004-693b2177b929 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:40.006098 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1249/2474] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:33:39 2022] POST /identity/v3/users => generated 292 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:33:40.012811 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e37a72-25bb-46e4-a10a-9687fc2eeb7f admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:40.035366 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e37a72-25bb-46e4-a10a-9687fc2eeb7f admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:40.042618 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be401d1-5201-4941-902e-f3767ce3a9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:40.043851 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be401d1-5201-4941-902e-f3767ce3a9ae admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:40.044354 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be401d1-5201-4941-902e-f3767ce3a9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:40.045015 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be401d1-5201-4941-902e-f3767ce3a9ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:40.046161 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be401d1-5201-4941-902e-f3767ce3a9ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:40.048372 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be401d1-5201-4941-902e-f3767ce3a9ae admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:40.066571 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:40.106610 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1250/2475] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:33:40 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:33:40.111861 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be401d1-5201-4941-902e-f3767ce3a9ae admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:40.111861 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-1be401d1-5201-4941-902e-f3767ce3a9ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:40.129409 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be401d1-5201-4941-902e-f3767ce3a9ae admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:40.137979 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3e3cd0-8d22-44e6-88e4-8efa4c3a25fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:40.139191 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e3cd0-8d22-44e6-88e4-8efa4c3a25fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:40.139554 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e3cd0-8d22-44e6-88e4-8efa4c3a25fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:40.140021 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e3cd0-8d22-44e6-88e4-8efa4c3a25fa admin admin] PATH_INFO: `/v3/users/0e634ab2f0944e24a087a10e36451f08` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:40.141727 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3e3cd0-8d22-44e6-88e4-8efa4c3a25fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e634ab2f0944e24a087a10e36451f08)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:40.144484 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3e3cd0-8d22-44e6-88e4-8efa4c3a25fa admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:40.177604 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3116322b-ca2c-4ee2-9004-693b2177b929 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:40.185143 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95326b10-c374-4279-b492-338aa50e92cc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938187967efe49be9c5389d1feef81ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:40.185829 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95326b10-c374-4279-b492-338aa50e92cc service placement] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:40.186111 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95326b10-c374-4279-b492-338aa50e92cc service placement] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:40.186387 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95326b10-c374-4279-b492-338aa50e92cc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:40.188191 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-95326b10-c374-4279-b492-338aa50e92cc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:40.216021 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95326b10-c374-4279-b492-338aa50e92cc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:40.217701 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95326b10-c374-4279-b492-338aa50e92cc service placement] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:40.232325 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1226/2476] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:33:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:33:40.242813 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95326b10-c374-4279-b492-338aa50e92cc service placement] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:40.282630 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95326b10-c374-4279-b492-338aa50e92cc service placement] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:40.296516 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5906a38-1af6-4a17-a140-059ea15f363e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:40.297370 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5906a38-1af6-4a17-a140-059ea15f363e admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:40.297640 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5906a38-1af6-4a17-a140-059ea15f363e admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:40.298047 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5906a38-1af6-4a17-a140-059ea15f363e admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:40.298896 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5906a38-1af6-4a17-a140-059ea15f363e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:40.300626 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5906a38-1af6-4a17-a140-059ea15f363e admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:40.322506 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1227/2477] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:33:40 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:33:40.323744 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5906a38-1af6-4a17-a140-059ea15f363e admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:40.343483 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5906a38-1af6-4a17-a140-059ea15f363e admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:40.349242 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b8f106-a81a-4b35-9e2c-b8285d95a9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:40.356378 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b8f106-a81a-4b35-9e2c-b8285d95a9c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:40.356378 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b8f106-a81a-4b35-9e2c-b8285d95a9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:40.356378 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b8f106-a81a-4b35-9e2c-b8285d95a9c8 admin admin] PATH_INFO: `/v3/projects/9f53aee86a83487285cea0f18c5e3a7b/users/35c8a1fa6f0d4223b0b553f06940b2cd/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:40.374928 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b8f106-a81a-4b35-9e2c-b8285d95a9c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f53aee86a83487285cea0f18c5e3a7b, user_id=35c8a1fa6f0d4223b0b553f06940b2cd, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:40.377986 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b8f106-a81a-4b35-9e2c-b8285d95a9c8 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:40.416660 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-cd3e3cd0-8d22-44e6-88e4-8efa4c3a25fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e634ab2f0944e24a087a10e36451f08'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:40.443360 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-cd3e3cd0-8d22-44e6-88e4-8efa4c3a25fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e634ab2f0944e24a087a10e36451f08'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:40.506126 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1228/2478] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:33:40 2022] PUT /identity/v3/projects/9f53aee86a83487285cea0f18c5e3a7b/users/35c8a1fa6f0d4223b0b553f06940b2cd/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:33:40.521081 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd4b389-a045-4bda-9782-80b0931096ca None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:40.521081 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd4b389-a045-4bda-9782-80b0931096ca None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:40.521750 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd4b389-a045-4bda-9782-80b0931096ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:40.538028 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-cd3e3cd0-8d22-44e6-88e4-8efa4c3a25fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e634ab2f0944e24a087a10e36451f08'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:40.538597 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-cd3e3cd0-8d22-44e6-88e4-8efa4c3a25fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e634ab2f0944e24a087a10e36451f08'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:40.560978 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-cd3e3cd0-8d22-44e6-88e4-8efa4c3a25fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e634ab2f0944e24a087a10e36451f08'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:40.574785 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1251/2479] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:33:40 2022] DELETE /identity/v3/users/0e634ab2f0944e24a087a10e36451f08 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:33:40.583888 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-9fd4b389-a045-4bda-9782-80b0931096ca None None] MFA Rules not processed for user `35c8a1fa6f0d4223b0b553f06940b2cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:33:40.586180 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-9fd4b389-a045-4bda-9782-80b0931096ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:40.729666 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1229/2480] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:33:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:33:40.731444 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3e3cd0-8d22-44e6-88e4-8efa4c3a25fa admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:40.733465 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-cd3e3cd0-8d22-44e6-88e4-8efa4c3a25fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:40.783692 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:40.856756 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3e3cd0-8d22-44e6-88e4-8efa4c3a25fa admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:40.866916 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1165aa81-b146-4d32-b546-9bbe777a88a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:40.870322 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1165aa81-b146-4d32-b546-9bbe777a88a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:40.873214 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1165aa81-b146-4d32-b546-9bbe777a88a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:40.878294 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1165aa81-b146-4d32-b546-9bbe777a88a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:40.880677 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1165aa81-b146-4d32-b546-9bbe777a88a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:40.885942 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1165aa81-b146-4d32-b546-9bbe777a88a8 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:40.954296 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd4b389-a045-4bda-9782-80b0931096ca None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:40.955997 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-9fd4b389-a045-4bda-9782-80b0931096ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:40.983139 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd4b389-a045-4bda-9782-80b0931096ca None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:40.988371 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fa42cb-f9fc-46f9-a955-88be37a280d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:40.989820 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fa42cb-f9fc-46f9-a955-88be37a280d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:40.990127 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fa42cb-f9fc-46f9-a955-88be37a280d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:40.990407 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fa42cb-f9fc-46f9-a955-88be37a280d5 admin admin] PATH_INFO: `/v3/projects/1a1786032e9449728e36f679515ced6d` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:40.991887 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fa42cb-f9fc-46f9-a955-88be37a280d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a1786032e9449728e36f679515ced6d)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:40.994847 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fa42cb-f9fc-46f9-a955-88be37a280d5 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:41.028865 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1252/2481] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:33:40 2022] POST /identity/v3/projects => generated 383 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:33:41.033304 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1165aa81-b146-4d32-b546-9bbe777a88a8 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:41.126777 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-17fa42cb-f9fc-46f9-a955-88be37a280d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a1786032e9449728e36f679515ced6d', 'request_id': 'req-17fa42cb-f9fc-46f9-a955-88be37a280d5'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:41.148519 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1165aa81-b146-4d32-b546-9bbe777a88a8 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:41.206280 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-17fa42cb-f9fc-46f9-a955-88be37a280d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a1786032e9449728e36f679515ced6d', 'request_id': 'req-17fa42cb-f9fc-46f9-a955-88be37a280d5'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:41.214358 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:41.215936 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-17fa42cb-f9fc-46f9-a955-88be37a280d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a1786032e9449728e36f679515ced6d', 'request_id': 'req-17fa42cb-f9fc-46f9-a955-88be37a280d5'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:41.247086 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-17fa42cb-f9fc-46f9-a955-88be37a280d5 admin admin] The token cache is being invalidate because project 1a1786032e9449728e36f679515ced6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:33:41.247410 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-17fa42cb-f9fc-46f9-a955-88be37a280d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:41.249148 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f30c1-1e0a-4736-a3ae-9f06f4e8b798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:41.250140 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f30c1-1e0a-4736-a3ae-9f06f4e8b798 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:41.250745 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f30c1-1e0a-4736-a3ae-9f06f4e8b798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:41.251186 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f30c1-1e0a-4736-a3ae-9f06f4e8b798 admin admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:41.252239 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91f30c1-1e0a-4736-a3ae-9f06f4e8b798 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:41.252602 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-17fa42cb-f9fc-46f9-a955-88be37a280d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:41.255055 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91f30c1-1e0a-4736-a3ae-9f06f4e8b798 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:41.255791 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1230/2482] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:33:40 2022] DELETE /identity/v3/projects/1a1786032e9449728e36f679515ced6d => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:33:41.267215 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fa42cb-f9fc-46f9-a955-88be37a280d5 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:41.270385 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-17fa42cb-f9fc-46f9-a955-88be37a280d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:41.290640 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fa42cb-f9fc-46f9-a955-88be37a280d5 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:41.296600 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42429c9-d942-4f5f-8920-c1672269312e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:41.297525 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42429c9-d942-4f5f-8920-c1672269312e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:41.297807 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42429c9-d942-4f5f-8920-c1672269312e admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:41.299021 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42429c9-d942-4f5f-8920-c1672269312e admin admin] PATH_INFO: `/v3/users/a043359d5cea4a2c938a1d1c781c8261` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:41.299490 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42429c9-d942-4f5f-8920-c1672269312e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a043359d5cea4a2c938a1d1c781c8261)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:41.302238 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42429c9-d942-4f5f-8920-c1672269312e admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:41.354533 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1253/2483] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:33:41 2022] POST /identity/v3/users => generated 298 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:33:41.360961 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f30c1-1e0a-4736-a3ae-9f06f4e8b798 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:41.363696 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-c91f30c1-1e0a-4736-a3ae-9f06f4e8b798 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:41.402241 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f30c1-1e0a-4736-a3ae-9f06f4e8b798 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:41.408712 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f0cb9c-bc04-45ff-aa22-77d96e780900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:41.409580 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0cb9c-bc04-45ff-aa22-77d96e780900 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:41.409962 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0cb9c-bc04-45ff-aa22-77d96e780900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:41.410486 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0cb9c-bc04-45ff-aa22-77d96e780900 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:41.411505 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f0cb9c-bc04-45ff-aa22-77d96e780900 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:41.413297 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f0cb9c-bc04-45ff-aa22-77d96e780900 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:41.432450 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1254/2484] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:33:41.434887 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f0cb9c-bc04-45ff-aa22-77d96e780900 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:41.447022 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f0cb9c-bc04-45ff-aa22-77d96e780900 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:41.452624 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21936c2b-33ed-40cb-9dec-d086e935066b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:41.453683 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21936c2b-33ed-40cb-9dec-d086e935066b admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:41.454098 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21936c2b-33ed-40cb-9dec-d086e935066b admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:41.454857 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21936c2b-33ed-40cb-9dec-d086e935066b admin admin] PATH_INFO: `/v3/projects/d5b0849730ac41839b6e44e6187a9708/users/2a64df56c13b412686b6e791a58b4ce2/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:41.473275 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21936c2b-33ed-40cb-9dec-d086e935066b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5b0849730ac41839b6e44e6187a9708, user_id=2a64df56c13b412686b6e791a58b4ce2, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:41.476752 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21936c2b-33ed-40cb-9dec-d086e935066b admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:41.524897 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-c42429c9-d942-4f5f-8920-c1672269312e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a043359d5cea4a2c938a1d1c781c8261'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:41.542181 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-c42429c9-d942-4f5f-8920-c1672269312e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a043359d5cea4a2c938a1d1c781c8261'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:41.543393 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-c42429c9-d942-4f5f-8920-c1672269312e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a043359d5cea4a2c938a1d1c781c8261'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:41.551997 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-c42429c9-d942-4f5f-8920-c1672269312e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a043359d5cea4a2c938a1d1c781c8261'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:41.553182 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-c42429c9-d942-4f5f-8920-c1672269312e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a043359d5cea4a2c938a1d1c781c8261'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:41.557758 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1255/2485] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:33:41 2022] PUT /identity/v3/projects/d5b0849730ac41839b6e44e6187a9708/users/2a64df56c13b412686b6e791a58b4ce2/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:33:41.560029 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21936c2b-33ed-40cb-9dec-d086e935066b admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:41.562463 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-21936c2b-33ed-40cb-9dec-d086e935066b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:41.582599 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:41.632947 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1231/2486] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:33:41 2022] DELETE /identity/v3/users/a043359d5cea4a2c938a1d1c781c8261 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:33:41.663239 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21936c2b-33ed-40cb-9dec-d086e935066b admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:41.762974 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a34c81-b303-43d4-87e7-3ba5dd30c746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:41.763997 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a34c81-b303-43d4-87e7-3ba5dd30c746 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:41.764603 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a34c81-b303-43d4-87e7-3ba5dd30c746 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:41.765080 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a34c81-b303-43d4-87e7-3ba5dd30c746 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:41.765984 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a34c81-b303-43d4-87e7-3ba5dd30c746 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:41.768339 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a34c81-b303-43d4-87e7-3ba5dd30c746 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:41.787209 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1256/2487] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:33:41.788040 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42429c9-d942-4f5f-8920-c1672269312e admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:41.802535 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42429c9-d942-4f5f-8920-c1672269312e admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:41.833573 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e0e09d-b8e2-4985-bed7-043c3fcec724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:41.834605 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e0e09d-b8e2-4985-bed7-043c3fcec724 admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:41.834896 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e0e09d-b8e2-4985-bed7-043c3fcec724 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:41.835171 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e0e09d-b8e2-4985-bed7-043c3fcec724 admin admin] PATH_INFO: `/v3/projects/d5b0849730ac41839b6e44e6187a9708/users/2a64df56c13b412686b6e791a58b4ce2/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:41.838796 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e0e09d-b8e2-4985-bed7-043c3fcec724 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5b0849730ac41839b6e44e6187a9708, user_id=2a64df56c13b412686b6e791a58b4ce2, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:41.843302 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e0e09d-b8e2-4985-bed7-043c3fcec724 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:41.870484 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1232/2488] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:33:41 2022] PUT /identity/v3/projects/d5b0849730ac41839b6e44e6187a9708/users/2a64df56c13b412686b6e791a58b4ce2/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:33:41.883495 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec5853-5356-4817-b7c8-fca1fdb89e8e None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:41.883495 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec5853-5356-4817-b7c8-fca1fdb89e8e None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:41.883495 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec5853-5356-4817-b7c8-fca1fdb89e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:41.919616 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-f3ec5853-5356-4817-b7c8-fca1fdb89e8e None None] MFA Rules not processed for user `2a64df56c13b412686b6e791a58b4ce2`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:33:41.922212 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-f3ec5853-5356-4817-b7c8-fca1fdb89e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:41.948464 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e0e09d-b8e2-4985-bed7-043c3fcec724 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:41.949125 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-97e0e09d-b8e2-4985-bed7-043c3fcec724 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:42.047871 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e0e09d-b8e2-4985-bed7-043c3fcec724 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:42.057185 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426dde1c-2edd-411e-a5e3-aff5c0f94e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:42.061604 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426dde1c-2edd-411e-a5e3-aff5c0f94e57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:42.061604 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426dde1c-2edd-411e-a5e3-aff5c0f94e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:42.061604 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426dde1c-2edd-411e-a5e3-aff5c0f94e57 admin admin] PATH_INFO: `/v3/projects/f8d71f4807a147e4b86450665a2d24a5` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:42.061604 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426dde1c-2edd-411e-a5e3-aff5c0f94e57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8d71f4807a147e4b86450665a2d24a5)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:42.063410 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426dde1c-2edd-411e-a5e3-aff5c0f94e57 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:42.128811 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1257/2489] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:33:41 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:33:42.156559 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ec5853-5356-4817-b7c8-fca1fdb89e8e None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:42.158258 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-f3ec5853-5356-4817-b7c8-fca1fdb89e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:42.213863 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-426dde1c-2edd-411e-a5e3-aff5c0f94e57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8d71f4807a147e4b86450665a2d24a5', 'request_id': 'req-426dde1c-2edd-411e-a5e3-aff5c0f94e57'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:42.218596 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:42.225556 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:42.246127 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ec5853-5356-4817-b7c8-fca1fdb89e8e None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:42.316348 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe1f2c-3a4a-4711-8d15-8e479864f91a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:42.317452 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe1f2c-3a4a-4711-8d15-8e479864f91a service cinder] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:42.317989 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe1f2c-3a4a-4711-8d15-8e479864f91a service cinder] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:42.318446 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe1f2c-3a4a-4711-8d15-8e479864f91a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:42.320574 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-426dde1c-2edd-411e-a5e3-aff5c0f94e57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8d71f4807a147e4b86450665a2d24a5', 'request_id': 'req-426dde1c-2edd-411e-a5e3-aff5c0f94e57'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:42.324874 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-67fe1f2c-3a4a-4711-8d15-8e479864f91a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:42.331935 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-426dde1c-2edd-411e-a5e3-aff5c0f94e57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8d71f4807a147e4b86450665a2d24a5', 'request_id': 'req-426dde1c-2edd-411e-a5e3-aff5c0f94e57'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:42.354347 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-426dde1c-2edd-411e-a5e3-aff5c0f94e57 admin admin] The token cache is being invalidate because project f8d71f4807a147e4b86450665a2d24a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:33:42.354904 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-426dde1c-2edd-411e-a5e3-aff5c0f94e57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:42.355572 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-426dde1c-2edd-411e-a5e3-aff5c0f94e57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:42.357548 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1233/2490] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:33:41 2022] DELETE /identity/v3/projects/f8d71f4807a147e4b86450665a2d24a5 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:33:42.438543 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fe1f2c-3a4a-4711-8d15-8e479864f91a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:42.440860 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fe1f2c-3a4a-4711-8d15-8e479864f91a service cinder] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:42.463458 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1258/2491] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:33:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:33:48.605435 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426dde1c-2edd-411e-a5e3-aff5c0f94e57 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:48.605435 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-426dde1c-2edd-411e-a5e3-aff5c0f94e57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:48.619573 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426dde1c-2edd-411e-a5e3-aff5c0f94e57 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:48.623782 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abefeaa-21fc-456b-bdbb-b949e7f0f08d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:48.624495 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abefeaa-21fc-456b-bdbb-b949e7f0f08d service cinder] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:48.624786 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abefeaa-21fc-456b-bdbb-b949e7f0f08d service cinder] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:48.625068 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abefeaa-21fc-456b-bdbb-b949e7f0f08d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:48.626949 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-2abefeaa-21fc-456b-bdbb-b949e7f0f08d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:48.700407 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abefeaa-21fc-456b-bdbb-b949e7f0f08d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:48.703090 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abefeaa-21fc-456b-bdbb-b949e7f0f08d service cinder] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:48.721084 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1234/2492] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:33:48 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:33:52.449682 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=50 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:54.893620 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe1f2c-3a4a-4711-8d15-8e479864f91a service cinder] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:54.967917 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe1f2c-3a4a-4711-8d15-8e479864f91a service cinder] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:54.972548 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef8439c-7faa-48a7-9b49-b802ed5b58c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:54.973424 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef8439c-7faa-48a7-9b49-b802ed5b58c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:54.973818 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef8439c-7faa-48a7-9b49-b802ed5b58c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:54.973971 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef8439c-7faa-48a7-9b49-b802ed5b58c6 admin admin] PATH_INFO: `/v3/users/35c8a1fa6f0d4223b0b553f06940b2cd` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:54.975190 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef8439c-7faa-48a7-9b49-b802ed5b58c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35c8a1fa6f0d4223b0b553f06940b2cd)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:54.977165 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef8439c-7faa-48a7-9b49-b802ed5b58c6 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:55.234702 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-8ef8439c-7faa-48a7-9b49-b802ed5b58c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35c8a1fa6f0d4223b0b553f06940b2cd'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:55.255209 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-8ef8439c-7faa-48a7-9b49-b802ed5b58c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35c8a1fa6f0d4223b0b553f06940b2cd'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:55.342596 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-8ef8439c-7faa-48a7-9b49-b802ed5b58c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35c8a1fa6f0d4223b0b553f06940b2cd'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:55.343396 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-8ef8439c-7faa-48a7-9b49-b802ed5b58c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35c8a1fa6f0d4223b0b553f06940b2cd'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:55.352250 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-8ef8439c-7faa-48a7-9b49-b802ed5b58c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35c8a1fa6f0d4223b0b553f06940b2cd'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:55.359520 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1259/2493] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:33:54 2022] DELETE /identity/v3/users/35c8a1fa6f0d4223b0b553f06940b2cd => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:33:55.364136 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b040ac-bdbf-4ee0-8aa8-459b171b809a None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:55.364707 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b040ac-bdbf-4ee0-8aa8-459b171b809a None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:55.364809 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b040ac-bdbf-4ee0-8aa8-459b171b809a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:55.385385 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-a5b040ac-bdbf-4ee0-8aa8-459b171b809a None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:33:55.388931 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-a5b040ac-bdbf-4ee0-8aa8-459b171b809a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:55.462758 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1235/2494] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:33:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:33:55.472801 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef8439c-7faa-48a7-9b49-b802ed5b58c6 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:55.474318 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-8ef8439c-7faa-48a7-9b49-b802ed5b58c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:55.544279 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef8439c-7faa-48a7-9b49-b802ed5b58c6 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:55.548825 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c0a5d0-b83f-4fc3-be38-47e504cf79c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:55.549499 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c0a5d0-b83f-4fc3-be38-47e504cf79c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:55.549849 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c0a5d0-b83f-4fc3-be38-47e504cf79c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:55.550109 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c0a5d0-b83f-4fc3-be38-47e504cf79c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:55.563682 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c0a5d0-b83f-4fc3-be38-47e504cf79c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:55.565261 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c0a5d0-b83f-4fc3-be38-47e504cf79c3 service neutron] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:55.578898 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1260/2495] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:33:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:33:56.109410 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b040ac-bdbf-4ee0-8aa8-459b171b809a None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:56.113600 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-a5b040ac-bdbf-4ee0-8aa8-459b171b809a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:56.125454 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b040ac-bdbf-4ee0-8aa8-459b171b809a None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:56.130768 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500cfdf3-353a-456a-8914-b7ef2f245810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:56.131660 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500cfdf3-353a-456a-8914-b7ef2f245810 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:56.131958 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500cfdf3-353a-456a-8914-b7ef2f245810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:56.132232 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500cfdf3-353a-456a-8914-b7ef2f245810 admin admin] PATH_INFO: `/v3/projects/9f53aee86a83487285cea0f18c5e3a7b` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:56.133460 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500cfdf3-353a-456a-8914-b7ef2f245810 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f53aee86a83487285cea0f18c5e3a7b)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:56.135602 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500cfdf3-353a-456a-8914-b7ef2f245810 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:56.241221 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-500cfdf3-353a-456a-8914-b7ef2f245810 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f53aee86a83487285cea0f18c5e3a7b', 'request_id': 'req-500cfdf3-353a-456a-8914-b7ef2f245810'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:56.312804 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-500cfdf3-353a-456a-8914-b7ef2f245810 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f53aee86a83487285cea0f18c5e3a7b', 'request_id': 'req-500cfdf3-353a-456a-8914-b7ef2f245810'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:56.319529 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-500cfdf3-353a-456a-8914-b7ef2f245810 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f53aee86a83487285cea0f18c5e3a7b', 'request_id': 'req-500cfdf3-353a-456a-8914-b7ef2f245810'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:56.327108 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:56.331690 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-500cfdf3-353a-456a-8914-b7ef2f245810 admin admin] The token cache is being invalidate because project 9f53aee86a83487285cea0f18c5e3a7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:33:56.334130 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-500cfdf3-353a-456a-8914-b7ef2f245810 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:56.334130 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-500cfdf3-353a-456a-8914-b7ef2f245810 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:56.334360 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1236/2496] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:33:56 2022] DELETE /identity/v3/projects/9f53aee86a83487285cea0f18c5e3a7b => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:33:56.338326 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c0a5d0-b83f-4fc3-be38-47e504cf79c3 service neutron] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:56.348859 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c0a5d0-b83f-4fc3-be38-47e504cf79c3 service neutron] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:56.369046 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:56.413640 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac059306-92be-4770-8f97-44001327d6ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:56.414626 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac059306-92be-4770-8f97-44001327d6ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:56.415070 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac059306-92be-4770-8f97-44001327d6ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:56.415417 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac059306-92be-4770-8f97-44001327d6ce admin admin] PATH_INFO: `/v3/users/2a64df56c13b412686b6e791a58b4ce2` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:56.418354 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac059306-92be-4770-8f97-44001327d6ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a64df56c13b412686b6e791a58b4ce2)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:56.420772 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac059306-92be-4770-8f97-44001327d6ce admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:56.623705 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-ac059306-92be-4770-8f97-44001327d6ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a64df56c13b412686b6e791a58b4ce2'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:56.638996 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-ac059306-92be-4770-8f97-44001327d6ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a64df56c13b412686b6e791a58b4ce2'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:56.712931 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-ac059306-92be-4770-8f97-44001327d6ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a64df56c13b412686b6e791a58b4ce2'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:56.713636 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-ac059306-92be-4770-8f97-44001327d6ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a64df56c13b412686b6e791a58b4ce2'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:56.722317 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-ac059306-92be-4770-8f97-44001327d6ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a64df56c13b412686b6e791a58b4ce2'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:56.730050 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1261/2497] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:33:56 2022] DELETE /identity/v3/users/2a64df56c13b412686b6e791a58b4ce2 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:33:57.212074 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf2557-8b34-484c-a99b-4c6e38472cb2 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:57.212579 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf2557-8b34-484c-a99b-4c6e38472cb2 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:57.212825 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf2557-8b34-484c-a99b-4c6e38472cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:57.232131 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-09bf2557-8b34-484c-a99b-4c6e38472cb2 None None] MFA Rules not processed for user `fcfcd9f134e74b09bfdac337c81c8a07`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:33:57.233358 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-09bf2557-8b34-484c-a99b-4c6e38472cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:57.285142 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac059306-92be-4770-8f97-44001327d6ce admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:57.286860 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-ac059306-92be-4770-8f97-44001327d6ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:57.299383 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1237/2498] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Jun 21 18:33:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:33:57.320057 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf2557-8b34-484c-a99b-4c6e38472cb2 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:57.320057 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-09bf2557-8b34-484c-a99b-4c6e38472cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:57.366868 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac059306-92be-4770-8f97-44001327d6ce admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:57.373136 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f916acdb-83ec-43b1-97e3-c19dae966b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:57.374153 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f916acdb-83ec-43b1-97e3-c19dae966b43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:57.374506 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f916acdb-83ec-43b1-97e3-c19dae966b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:57.374786 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f916acdb-83ec-43b1-97e3-c19dae966b43 admin admin] PATH_INFO: `/v3/projects/d5b0849730ac41839b6e44e6187a9708` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:57.375989 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f916acdb-83ec-43b1-97e3-c19dae966b43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5b0849730ac41839b6e44e6187a9708)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:57.378197 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f916acdb-83ec-43b1-97e3-c19dae966b43 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:57.396934 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf2557-8b34-484c-a99b-4c6e38472cb2 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:57.403248 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eab0de3-564c-4adb-819b-b3b9682544bd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e0d25565f1543d9a7577288fc09119f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:57.403942 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab0de3-564c-4adb-819b-b3b9682544bd service swift] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:57.404252 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab0de3-564c-4adb-819b-b3b9682544bd service swift] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:57.404581 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab0de3-564c-4adb-819b-b3b9682544bd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:57.414087 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eab0de3-564c-4adb-819b-b3b9682544bd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:57.415692 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eab0de3-564c-4adb-819b-b3b9682544bd service swift] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:57.425063 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1238/2499] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Jun 21 18:33:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 21 18:33:57.547140 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:57.551764 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-f916acdb-83ec-43b1-97e3-c19dae966b43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5b0849730ac41839b6e44e6187a9708', 'request_id': 'req-f916acdb-83ec-43b1-97e3-c19dae966b43'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:57.559770 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-f916acdb-83ec-43b1-97e3-c19dae966b43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5b0849730ac41839b6e44e6187a9708', 'request_id': 'req-f916acdb-83ec-43b1-97e3-c19dae966b43'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:57.565782 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-f916acdb-83ec-43b1-97e3-c19dae966b43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5b0849730ac41839b6e44e6187a9708', 'request_id': 'req-f916acdb-83ec-43b1-97e3-c19dae966b43'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:57.670120 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-f916acdb-83ec-43b1-97e3-c19dae966b43 admin admin] The token cache is being invalidate because project d5b0849730ac41839b6e44e6187a9708 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:33:57.670120 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-f916acdb-83ec-43b1-97e3-c19dae966b43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:57.671400 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-f916acdb-83ec-43b1-97e3-c19dae966b43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:57.673905 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1262/2500] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:33:57 2022] DELETE /identity/v3/projects/d5b0849730ac41839b6e44e6187a9708 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:33:57.680825 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722ea741-889b-4c90-8f66-0430c1b044a1 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:57.681256 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722ea741-889b-4c90-8f66-0430c1b044a1 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:57.681690 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722ea741-889b-4c90-8f66-0430c1b044a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:57.723777 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-722ea741-889b-4c90-8f66-0430c1b044a1 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:33:57.726376 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-722ea741-889b-4c90-8f66-0430c1b044a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:57.830002 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1239/2501] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:33:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:33:57.839126 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d2693-30ee-4542-ae38-02e7237bad45 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:57.840261 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d2693-30ee-4542-ae38-02e7237bad45 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:57.840861 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d2693-30ee-4542-ae38-02e7237bad45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:57.875721 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-250d2693-30ee-4542-ae38-02e7237bad45 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:33:57.877036 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-250d2693-30ee-4542-ae38-02e7237bad45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:57.903298 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1263/2502] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:33:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:33:57.907769 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722ea741-889b-4c90-8f66-0430c1b044a1 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:57.923497 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722ea741-889b-4c90-8f66-0430c1b044a1 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:57.941891 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00360a92-cc79-4e68-80f7-a437b4cc1632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:57.942927 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00360a92-cc79-4e68-80f7-a437b4cc1632 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:57.943160 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00360a92-cc79-4e68-80f7-a437b4cc1632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:57.943442 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00360a92-cc79-4e68-80f7-a437b4cc1632 admin admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:57.944335 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00360a92-cc79-4e68-80f7-a437b4cc1632 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:33:57.944832 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00360a92-cc79-4e68-80f7-a437b4cc1632 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:57.946616 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00360a92-cc79-4e68-80f7-a437b4cc1632 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:57.981512 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1240/2503] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:33:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:33:57.983667 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250d2693-30ee-4542-ae38-02e7237bad45 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:58.003405 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250d2693-30ee-4542-ae38-02e7237bad45 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:58.017257 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d169a5bd-569b-4152-abe2-c3509a9fa616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:58.017957 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d169a5bd-569b-4152-abe2-c3509a9fa616 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:58.018246 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d169a5bd-569b-4152-abe2-c3509a9fa616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:58.018532 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d169a5bd-569b-4152-abe2-c3509a9fa616 admin admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:58.020054 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d169a5bd-569b-4152-abe2-c3509a9fa616 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:58.021278 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d169a5bd-569b-4152-abe2-c3509a9fa616 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:58.137436 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1264/2504] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:33:57 2022] POST /identity/v3/projects => generated 385 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:33:58.139292 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00360a92-cc79-4e68-80f7-a437b4cc1632 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:58.152985 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00360a92-cc79-4e68-80f7-a437b4cc1632 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:58.193873 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:58.296158 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20ad256-ff21-4a0f-b789-a945ed030e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:58.297459 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20ad256-ff21-4a0f-b789-a945ed030e84 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:58.297459 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20ad256-ff21-4a0f-b789-a945ed030e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:58.297751 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20ad256-ff21-4a0f-b789-a945ed030e84 admin admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:58.298241 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20ad256-ff21-4a0f-b789-a945ed030e84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:58.300351 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20ad256-ff21-4a0f-b789-a945ed030e84 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:58.442203 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1241/2505] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:33:58 2022] POST /identity/v3/users => generated 293 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:33:58.446647 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d169a5bd-569b-4152-abe2-c3509a9fa616 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:58.458473 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d169a5bd-569b-4152-abe2-c3509a9fa616 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:58.465231 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa0f741-d856-451e-96ac-5b0bd02786f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:58.466784 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa0f741-d856-451e-96ac-5b0bd02786f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:58.467072 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa0f741-d856-451e-96ac-5b0bd02786f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:58.467355 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa0f741-d856-451e-96ac-5b0bd02786f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:58.468110 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa0f741-d856-451e-96ac-5b0bd02786f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:58.469797 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa0f741-d856-451e-96ac-5b0bd02786f8 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:58.483770 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1265/2506] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:33:58.487976 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20ad256-ff21-4a0f-b789-a945ed030e84 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:58.501835 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20ad256-ff21-4a0f-b789-a945ed030e84 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:58.509087 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ca2854-928b-4d55-b87a-4c5d2968eca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:58.510056 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ca2854-928b-4d55-b87a-4c5d2968eca2 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:58.510348 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ca2854-928b-4d55-b87a-4c5d2968eca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:58.510623 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ca2854-928b-4d55-b87a-4c5d2968eca2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:58.511908 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ca2854-928b-4d55-b87a-4c5d2968eca2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:58.514412 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ca2854-928b-4d55-b87a-4c5d2968eca2 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:58.528546 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1242/2507] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:33:58.533136 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa0f741-d856-451e-96ac-5b0bd02786f8 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:58.544312 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa0f741-d856-451e-96ac-5b0bd02786f8 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:58.549332 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a505d87f-2d15-4cd9-a90e-8227c411087b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:58.550274 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a505d87f-2d15-4cd9-a90e-8227c411087b admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:58.550640 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a505d87f-2d15-4cd9-a90e-8227c411087b admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:58.550987 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a505d87f-2d15-4cd9-a90e-8227c411087b admin admin] PATH_INFO: `/v3/projects/250e36680fd642829b9d073b5c0f13c9/users/4feef5bc980c420681725fc28b85e007/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:58.575739 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a505d87f-2d15-4cd9-a90e-8227c411087b admin admin] RBAC: Authorizing `identity:create_grant(project_id=250e36680fd642829b9d073b5c0f13c9, user_id=4feef5bc980c420681725fc28b85e007, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:58.578421 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a505d87f-2d15-4cd9-a90e-8227c411087b admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:58.653573 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1266/2508] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:33:58 2022] PUT /identity/v3/projects/250e36680fd642829b9d073b5c0f13c9/users/4feef5bc980c420681725fc28b85e007/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:33:58.665589 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076a380c-b4f0-49ca-9192-dc57789699b3 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:58.666971 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076a380c-b4f0-49ca-9192-dc57789699b3 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:58.666971 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076a380c-b4f0-49ca-9192-dc57789699b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:58.702316 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-076a380c-b4f0-49ca-9192-dc57789699b3 None None] MFA Rules not processed for user `4feef5bc980c420681725fc28b85e007`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:33:58.703775 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-076a380c-b4f0-49ca-9192-dc57789699b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:58.809083 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1243/2509] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:33:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:33:58.828944 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a505d87f-2d15-4cd9-a90e-8227c411087b admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:58.831681 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-a505d87f-2d15-4cd9-a90e-8227c411087b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:58.939269 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a505d87f-2d15-4cd9-a90e-8227c411087b admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:58.945288 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e218e984-e429-4812-8a65-ec654f94bf21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:58.946539 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218e984-e429-4812-8a65-ec654f94bf21 service cinder] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:58.947001 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218e984-e429-4812-8a65-ec654f94bf21 service cinder] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:58.947469 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218e984-e429-4812-8a65-ec654f94bf21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:58.958671 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e218e984-e429-4812-8a65-ec654f94bf21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:58.960152 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e218e984-e429-4812-8a65-ec654f94bf21 service cinder] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:58.971947 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1267/2510] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:33:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:33:59.005262 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076a380c-b4f0-49ca-9192-dc57789699b3 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:59.113837 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076a380c-b4f0-49ca-9192-dc57789699b3 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:59.122604 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef76c2-2cef-4fb6-95b1-0a8c5c10ea71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:59.124123 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef76c2-2cef-4fb6-95b1-0a8c5c10ea71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:59.124733 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef76c2-2cef-4fb6-95b1-0a8c5c10ea71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:59.125230 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef76c2-2cef-4fb6-95b1-0a8c5c10ea71 admin admin] PATH_INFO: `/v3/users/4feef5bc980c420681725fc28b85e007` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:59.127491 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ef76c2-2cef-4fb6-95b1-0a8c5c10ea71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4feef5bc980c420681725fc28b85e007)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:59.138965 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ef76c2-2cef-4fb6-95b1-0a8c5c10ea71 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:59.302428 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:33:59.336217 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-72ef76c2-2cef-4fb6-95b1-0a8c5c10ea71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4feef5bc980c420681725fc28b85e007'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:59.365275 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-72ef76c2-2cef-4fb6-95b1-0a8c5c10ea71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4feef5bc980c420681725fc28b85e007'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:59.366333 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-72ef76c2-2cef-4fb6-95b1-0a8c5c10ea71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4feef5bc980c420681725fc28b85e007'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:59.376090 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-72ef76c2-2cef-4fb6-95b1-0a8c5c10ea71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4feef5bc980c420681725fc28b85e007'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:59.377226 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-72ef76c2-2cef-4fb6-95b1-0a8c5c10ea71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4feef5bc980c420681725fc28b85e007'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:33:59.448595 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1244/2511] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:33:59 2022] DELETE /identity/v3/users/4feef5bc980c420681725fc28b85e007 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:33:59.456166 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e91793-e7bf-4496-93f9-19ea7d9d35e7 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:59.456834 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e91793-e7bf-4496-93f9-19ea7d9d35e7 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:59.457216 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e91793-e7bf-4496-93f9-19ea7d9d35e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:59.483256 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-30e91793-e7bf-4496-93f9-19ea7d9d35e7 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:33:59.484719 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-30e91793-e7bf-4496-93f9-19ea7d9d35e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:59.603742 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1268/2512] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:33:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:33:59.621631 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef76c2-2cef-4fb6-95b1-0a8c5c10ea71 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:33:59.623728 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-72ef76c2-2cef-4fb6-95b1-0a8c5c10ea71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:33:59.705933 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef76c2-2cef-4fb6-95b1-0a8c5c10ea71 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:33:59.711275 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6839a66-eb05-4ebd-a902-c8a6e97afa64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:33:59.712376 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6839a66-eb05-4ebd-a902-c8a6e97afa64 service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:33:59.712879 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6839a66-eb05-4ebd-a902-c8a6e97afa64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:33:59.713333 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6839a66-eb05-4ebd-a902-c8a6e97afa64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:33:59.725486 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6839a66-eb05-4ebd-a902-c8a6e97afa64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:33:59.727054 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6839a66-eb05-4ebd-a902-c8a6e97afa64 service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:33:59.738975 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1245/2513] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:33:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:34:00.269768 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e91793-e7bf-4496-93f9-19ea7d9d35e7 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:00.271365 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-30e91793-e7bf-4496-93f9-19ea7d9d35e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:00.294329 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e91793-e7bf-4496-93f9-19ea7d9d35e7 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:00.321843 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e0c2d7-dcf7-45a5-a8c2-69a01d3388f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:00.322865 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e0c2d7-dcf7-45a5-a8c2-69a01d3388f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:00.323231 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e0c2d7-dcf7-45a5-a8c2-69a01d3388f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:00.323578 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e0c2d7-dcf7-45a5-a8c2-69a01d3388f8 admin admin] PATH_INFO: `/v3/projects/250e36680fd642829b9d073b5c0f13c9` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:00.332311 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e0c2d7-dcf7-45a5-a8c2-69a01d3388f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=250e36680fd642829b9d073b5c0f13c9)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:00.334454 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e0c2d7-dcf7-45a5-a8c2-69a01d3388f8 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:00.343008 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:34:00.469956 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-b3e0c2d7-dcf7-45a5-a8c2-69a01d3388f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '250e36680fd642829b9d073b5c0f13c9', 'request_id': 'req-b3e0c2d7-dcf7-45a5-a8c2-69a01d3388f8'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:00.483091 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-b3e0c2d7-dcf7-45a5-a8c2-69a01d3388f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '250e36680fd642829b9d073b5c0f13c9', 'request_id': 'req-b3e0c2d7-dcf7-45a5-a8c2-69a01d3388f8'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:00.489130 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-b3e0c2d7-dcf7-45a5-a8c2-69a01d3388f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '250e36680fd642829b9d073b5c0f13c9', 'request_id': 'req-b3e0c2d7-dcf7-45a5-a8c2-69a01d3388f8'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:00.577362 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-b3e0c2d7-dcf7-45a5-a8c2-69a01d3388f8 admin admin] The token cache is being invalidate because project 250e36680fd642829b9d073b5c0f13c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:34:00.578120 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-b3e0c2d7-dcf7-45a5-a8c2-69a01d3388f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:00.579140 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-b3e0c2d7-dcf7-45a5-a8c2-69a01d3388f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:00.581083 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1269/2514] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:34:00 2022] DELETE /identity/v3/projects/250e36680fd642829b9d073b5c0f13c9 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:34:00.587367 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c603b4-58fe-4aea-afd0-febc5d4526a7 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:00.588275 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c603b4-58fe-4aea-afd0-febc5d4526a7 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:00.588758 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c603b4-58fe-4aea-afd0-febc5d4526a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:00.609501 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-80c603b4-58fe-4aea-afd0-febc5d4526a7 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:34:00.612233 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-80c603b4-58fe-4aea-afd0-febc5d4526a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:00.719635 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1246/2515] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:34:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:34:00.721601 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eff242-58d8-4836-8e47-c1ebf110e3d7 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:00.721965 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eff242-58d8-4836-8e47-c1ebf110e3d7 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:00.722354 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eff242-58d8-4836-8e47-c1ebf110e3d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:00.747717 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-a1eff242-58d8-4836-8e47-c1ebf110e3d7 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:34:00.749180 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-a1eff242-58d8-4836-8e47-c1ebf110e3d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:00.757718 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1270/2516] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:34:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:34:00.763752 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c603b4-58fe-4aea-afd0-febc5d4526a7 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:00.774316 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c603b4-58fe-4aea-afd0-febc5d4526a7 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:00.780002 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3f8e8d-a5a4-4b01-acae-589e0dacab60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:00.780647 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f8e8d-a5a4-4b01-acae-589e0dacab60 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:00.780931 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f8e8d-a5a4-4b01-acae-589e0dacab60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:00.781239 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f8e8d-a5a4-4b01-acae-589e0dacab60 admin admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:00.781977 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3f8e8d-a5a4-4b01-acae-589e0dacab60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:34:00.782304 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3f8e8d-a5a4-4b01-acae-589e0dacab60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:00.783974 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3f8e8d-a5a4-4b01-acae-589e0dacab60 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:00.802064 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1247/2517] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:34:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:34:00.804805 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eff242-58d8-4836-8e47-c1ebf110e3d7 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:00.816653 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eff242-58d8-4836-8e47-c1ebf110e3d7 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:00.827693 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ce746-8b39-419d-ae64-517ee3eb7c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:00.828747 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ce746-8b39-419d-ae64-517ee3eb7c8c admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:00.829261 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ce746-8b39-419d-ae64-517ee3eb7c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:00.829744 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ce746-8b39-419d-ae64-517ee3eb7c8c admin admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:00.830666 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810ce746-8b39-419d-ae64-517ee3eb7c8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:00.832694 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810ce746-8b39-419d-ae64-517ee3eb7c8c admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:00.939502 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1271/2518] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:34:00 2022] POST /identity/v3/projects => generated 365 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:34:00.943770 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3f8e8d-a5a4-4b01-acae-589e0dacab60 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:00.956081 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3f8e8d-a5a4-4b01-acae-589e0dacab60 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:01.000189 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:34:01.031019 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd198e2-b9d7-468d-8252-3136787d1f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:01.031944 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd198e2-b9d7-468d-8252-3136787d1f1a admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:01.032200 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd198e2-b9d7-468d-8252-3136787d1f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:01.032524 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd198e2-b9d7-468d-8252-3136787d1f1a admin admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:01.033828 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd198e2-b9d7-468d-8252-3136787d1f1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:01.036735 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd198e2-b9d7-468d-8252-3136787d1f1a admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:01.147908 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1248/2519] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:34:00 2022] POST /identity/v3/users => generated 283 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:34:01.153650 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ce746-8b39-419d-ae64-517ee3eb7c8c admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:01.165382 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ce746-8b39-419d-ae64-517ee3eb7c8c admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:01.173142 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6372d949-ba7d-4a07-aa15-e7ea386decf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:01.174497 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6372d949-ba7d-4a07-aa15-e7ea386decf7 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:01.174821 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6372d949-ba7d-4a07-aa15-e7ea386decf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:01.175081 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6372d949-ba7d-4a07-aa15-e7ea386decf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:01.175998 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6372d949-ba7d-4a07-aa15-e7ea386decf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:01.180027 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6372d949-ba7d-4a07-aa15-e7ea386decf7 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:01.201175 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1272/2520] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:34:01 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:34:01.205600 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd198e2-b9d7-468d-8252-3136787d1f1a admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:01.226466 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd198e2-b9d7-468d-8252-3136787d1f1a admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:01.232752 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d82a37f-6e66-4ca3-84e4-5044defd90e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:01.233303 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d82a37f-6e66-4ca3-84e4-5044defd90e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:01.233600 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d82a37f-6e66-4ca3-84e4-5044defd90e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:01.233896 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d82a37f-6e66-4ca3-84e4-5044defd90e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:01.234609 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d82a37f-6e66-4ca3-84e4-5044defd90e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:01.236304 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d82a37f-6e66-4ca3-84e4-5044defd90e1 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:01.248269 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1249/2521] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:34:01 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:34:01.252517 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6372d949-ba7d-4a07-aa15-e7ea386decf7 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:01.264431 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6372d949-ba7d-4a07-aa15-e7ea386decf7 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:01.270609 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad56b8-75cf-41d9-8fa5-b06cefb5fc8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:01.271587 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad56b8-75cf-41d9-8fa5-b06cefb5fc8a admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:01.272066 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad56b8-75cf-41d9-8fa5-b06cefb5fc8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:01.272356 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad56b8-75cf-41d9-8fa5-b06cefb5fc8a admin admin] PATH_INFO: `/v3/projects/f4b3a84f1b0c49c0a9555ab23e096909/users/0f72def174a74b29b0203474c3dc2e69/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:01.290787 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baad56b8-75cf-41d9-8fa5-b06cefb5fc8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4b3a84f1b0c49c0a9555ab23e096909, user_id=0f72def174a74b29b0203474c3dc2e69, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:01.294355 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baad56b8-75cf-41d9-8fa5-b06cefb5fc8a admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:01.364361 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1273/2522] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:34:01 2022] PUT /identity/v3/projects/f4b3a84f1b0c49c0a9555ab23e096909/users/0f72def174a74b29b0203474c3dc2e69/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:34:01.376499 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5771630-6edd-47dd-af26-0e746071bcc5 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:01.377001 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5771630-6edd-47dd-af26-0e746071bcc5 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:01.377441 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5771630-6edd-47dd-af26-0e746071bcc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:01.412345 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-d5771630-6edd-47dd-af26-0e746071bcc5 None None] MFA Rules not processed for user `0f72def174a74b29b0203474c3dc2e69`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:34:01.414744 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-d5771630-6edd-47dd-af26-0e746071bcc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:01.539194 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1250/2523] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:34:01 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:34:01.553974 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad56b8-75cf-41d9-8fa5-b06cefb5fc8a admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:01.555729 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-baad56b8-75cf-41d9-8fa5-b06cefb5fc8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:01.648306 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad56b8-75cf-41d9-8fa5-b06cefb5fc8a admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:01.655631 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19690c55-f14d-4b3b-bb1b-2d367b82112d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:01.656681 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19690c55-f14d-4b3b-bb1b-2d367b82112d service cinder] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:01.657028 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19690c55-f14d-4b3b-bb1b-2d367b82112d service cinder] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:01.657316 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19690c55-f14d-4b3b-bb1b-2d367b82112d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:01.669656 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19690c55-f14d-4b3b-bb1b-2d367b82112d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:01.671182 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19690c55-f14d-4b3b-bb1b-2d367b82112d service cinder] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:01.691565 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1274/2524] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:34:01 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:34:11.535648 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=49,keystone:INSERT=3 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:34:11.676888 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=75,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:34:12.806246 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5771630-6edd-47dd-af26-0e746071bcc5 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:12.807643 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-d5771630-6edd-47dd-af26-0e746071bcc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:12.871418 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5771630-6edd-47dd-af26-0e746071bcc5 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:12.875557 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03b121a-6b8e-4a12-9ea3-8f025f7d236a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:12.876251 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b121a-6b8e-4a12-9ea3-8f025f7d236a service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:12.876568 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b121a-6b8e-4a12-9ea3-8f025f7d236a service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:12.876852 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b121a-6b8e-4a12-9ea3-8f025f7d236a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:12.878958 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-a03b121a-6b8e-4a12-9ea3-8f025f7d236a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:12.891040 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03b121a-6b8e-4a12-9ea3-8f025f7d236a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:12.892548 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03b121a-6b8e-4a12-9ea3-8f025f7d236a service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:12.901322 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1251/2525] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:34:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:34:13.241062 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a3aaf-f516-413f-9ccd-1b89994b0e69 None None] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:13.241756 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a3aaf-f516-413f-9ccd-1b89994b0e69 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:13.242032 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a3aaf-f516-413f-9ccd-1b89994b0e69 None None] PATH_INFO: `/` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:13.245133 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1275/2526] 10.222.0.23 () {54 vars in 921 bytes} [Tue Jun 21 18:34:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:34:13.251737 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b02721-d996-4156-9bf3-93c0067ae1d0 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:13.252115 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b02721-d996-4156-9bf3-93c0067ae1d0 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:13.252515 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b02721-d996-4156-9bf3-93c0067ae1d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:13.275294 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-a4b02721-d996-4156-9bf3-93c0067ae1d0 None None] MFA Rules not processed for user `59c7f07e58ba4db9a7d402d8f3da1e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:34:13.276522 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-a4b02721-d996-4156-9bf3-93c0067ae1d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:13.283815 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1252/2527] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Jun 21 18:34:13 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:34:13.303435 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35a3aaf-f516-413f-9ccd-1b89994b0e69 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:13.305205 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-c35a3aaf-f516-413f-9ccd-1b89994b0e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:13.325223 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35a3aaf-f516-413f-9ccd-1b89994b0e69 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:13.330318 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2332f4c4-faec-4778-9121-846fa3af27e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:13.331391 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332f4c4-faec-4778-9121-846fa3af27e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:13.331870 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332f4c4-faec-4778-9121-846fa3af27e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:13.332150 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332f4c4-faec-4778-9121-846fa3af27e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:13.340762 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2332f4c4-faec-4778-9121-846fa3af27e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:13.343377 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2332f4c4-faec-4778-9121-846fa3af27e5 service neutron] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:13.354533 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1276/2528] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:34:13 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:34:13.439975 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a54d6-fc1d-448f-83ee-7e4882280f2e None None] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:13.440272 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a54d6-fc1d-448f-83ee-7e4882280f2e None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:13.440655 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a54d6-fc1d-448f-83ee-7e4882280f2e None None] PATH_INFO: `/` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:13.442107 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1253/2529] 10.222.0.23 () {54 vars in 921 bytes} [Tue Jun 21 18:34:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:34:13.447442 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa0a5c-38ac-4175-a1b6-ccd20848230c None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:13.447761 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa0a5c-38ac-4175-a1b6-ccd20848230c None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:13.448041 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa0a5c-38ac-4175-a1b6-ccd20848230c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:13.467124 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-9caa0a5c-38ac-4175-a1b6-ccd20848230c None None] MFA Rules not processed for user `904255dd3c1f406eb10f6fc65f9cd098`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:34:13.468358 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-9caa0a5c-38ac-4175-a1b6-ccd20848230c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:13.536231 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1277/2530] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Jun 21 18:34:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:34:13.548040 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a54d6-fc1d-448f-83ee-7e4882280f2e None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:13.559203 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a54d6-fc1d-448f-83ee-7e4882280f2e None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:13.563436 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d331e6-8764-40e7-80b5-30d0ea56b421 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:13.564081 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d331e6-8764-40e7-80b5-30d0ea56b421 service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:13.564365 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d331e6-8764-40e7-80b5-30d0ea56b421 service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:13.564733 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d331e6-8764-40e7-80b5-30d0ea56b421 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:13.575159 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d331e6-8764-40e7-80b5-30d0ea56b421 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:13.576736 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d331e6-8764-40e7-80b5-30d0ea56b421 service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:13.587850 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1254/2531] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:34:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:34:16.538338 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e7a2a-5666-4b17-bcfc-57552bddffee None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:16.539904 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e7a2a-5666-4b17-bcfc-57552bddffee None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:16.540568 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e7a2a-5666-4b17-bcfc-57552bddffee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:16.564864 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-781e7a2a-5666-4b17-bcfc-57552bddffee None None] MFA Rules not processed for user `fcfcd9f134e74b09bfdac337c81c8a07`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:34:16.567172 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-781e7a2a-5666-4b17-bcfc-57552bddffee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:16.637518 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1278/2532] 10.222.0.23 () {58 vars in 967 bytes} [Tue Jun 21 18:34:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:34:16.650465 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d331e6-8764-40e7-80b5-30d0ea56b421 service neutron] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:16.651928 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-96d331e6-8764-40e7-80b5-30d0ea56b421 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:16.738290 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d331e6-8764-40e7-80b5-30d0ea56b421 service neutron] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:16.744080 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9196b86a-c9f7-4734-8e0f-bb87df66f4ae service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e0d25565f1543d9a7577288fc09119f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:16.744835 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9196b86a-c9f7-4734-8e0f-bb87df66f4ae service swift] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:16.745052 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9196b86a-c9f7-4734-8e0f-bb87df66f4ae service swift] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:16.745330 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9196b86a-c9f7-4734-8e0f-bb87df66f4ae service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:16.756577 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9196b86a-c9f7-4734-8e0f-bb87df66f4ae service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:16.758897 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9196b86a-c9f7-4734-8e0f-bb87df66f4ae service swift] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:16.769414 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1255/2533] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Jun 21 18:34:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 18:34:26.630545 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=62 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:34:26.766840 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=67 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:34:37.608415 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781e7a2a-5666-4b17-bcfc-57552bddffee None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:37.618059 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781e7a2a-5666-4b17-bcfc-57552bddffee None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:37.622085 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc021ed-eecd-4371-8578-bd106efffde8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:37.622676 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc021ed-eecd-4371-8578-bd106efffde8 service neutron] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:37.622962 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc021ed-eecd-4371-8578-bd106efffde8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:37.623241 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc021ed-eecd-4371-8578-bd106efffde8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:37.625296 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-3bc021ed-eecd-4371-8578-bd106efffde8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:37.691801 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc021ed-eecd-4371-8578-bd106efffde8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:37.693259 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc021ed-eecd-4371-8578-bd106efffde8 service neutron] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:37.710878 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1279/2534] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:34:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:34:41.546704 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9196b86a-c9f7-4734-8e0f-bb87df66f4ae service swift] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:41.551853 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-9196b86a-c9f7-4734-8e0f-bb87df66f4ae service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:41.657038 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9196b86a-c9f7-4734-8e0f-bb87df66f4ae service swift] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:41.663779 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa0036f-bf7c-46d8-898c-a1ab1faf37a5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938187967efe49be9c5389d1feef81ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:41.664464 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa0036f-bf7c-46d8-898c-a1ab1faf37a5 service placement] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:41.664753 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa0036f-bf7c-46d8-898c-a1ab1faf37a5 service placement] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:41.665031 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa0036f-bf7c-46d8-898c-a1ab1faf37a5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:41.666793 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-daa0036f-bf7c-46d8-898c-a1ab1faf37a5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:41.679040 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa0036f-bf7c-46d8-898c-a1ab1faf37a5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:41.680623 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa0036f-bf7c-46d8-898c-a1ab1faf37a5 service placement] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:41.691777 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1256/2535] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:34:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:34:41.810956 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc021ed-eecd-4371-8578-bd106efffde8 service neutron] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:41.812592 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-3bc021ed-eecd-4371-8578-bd106efffde8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:41.825968 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc021ed-eecd-4371-8578-bd106efffde8 service neutron] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:41.831718 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a207cbb6-4da7-4acc-996b-b704d1f34753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:41.834716 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a207cbb6-4da7-4acc-996b-b704d1f34753 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:41.834716 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a207cbb6-4da7-4acc-996b-b704d1f34753 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:41.834716 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a207cbb6-4da7-4acc-996b-b704d1f34753 admin admin] PATH_INFO: `/v3/users/5c21b9f596da493fac827d3573833ed1` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:41.836845 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a207cbb6-4da7-4acc-996b-b704d1f34753 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c21b9f596da493fac827d3573833ed1)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:41.838914 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a207cbb6-4da7-4acc-996b-b704d1f34753 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:42.103685 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-a207cbb6-4da7-4acc-996b-b704d1f34753 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c21b9f596da493fac827d3573833ed1'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:42.116071 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-a207cbb6-4da7-4acc-996b-b704d1f34753 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c21b9f596da493fac827d3573833ed1'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:42.214721 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-a207cbb6-4da7-4acc-996b-b704d1f34753 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c21b9f596da493fac827d3573833ed1'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:42.215415 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-a207cbb6-4da7-4acc-996b-b704d1f34753 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c21b9f596da493fac827d3573833ed1'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:42.224107 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-a207cbb6-4da7-4acc-996b-b704d1f34753 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c21b9f596da493fac827d3573833ed1'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:42.230829 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1280/2536] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:34:41 2022] DELETE /identity/v3/users/5c21b9f596da493fac827d3573833ed1 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:34:42.524076 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa0036f-bf7c-46d8-898c-a1ab1faf37a5 service placement] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:42.535900 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa0036f-bf7c-46d8-898c-a1ab1faf37a5 service placement] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:42.600789 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f002b7-64c2-470d-95fd-8026eb8435cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:42.601673 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f002b7-64c2-470d-95fd-8026eb8435cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:42.601956 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f002b7-64c2-470d-95fd-8026eb8435cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:42.602266 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f002b7-64c2-470d-95fd-8026eb8435cb admin admin] PATH_INFO: `/v3/projects/48dc636c17544593aaf2a2409c13e540` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:42.603482 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f002b7-64c2-470d-95fd-8026eb8435cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=48dc636c17544593aaf2a2409c13e540)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:42.605451 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f002b7-64c2-470d-95fd-8026eb8435cb admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:42.759394 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-33f002b7-64c2-470d-95fd-8026eb8435cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48dc636c17544593aaf2a2409c13e540', 'request_id': 'req-33f002b7-64c2-470d-95fd-8026eb8435cb'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:42.832417 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-33f002b7-64c2-470d-95fd-8026eb8435cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48dc636c17544593aaf2a2409c13e540', 'request_id': 'req-33f002b7-64c2-470d-95fd-8026eb8435cb'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:42.839010 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-33f002b7-64c2-470d-95fd-8026eb8435cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48dc636c17544593aaf2a2409c13e540', 'request_id': 'req-33f002b7-64c2-470d-95fd-8026eb8435cb'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:42.852098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-33f002b7-64c2-470d-95fd-8026eb8435cb admin admin] The token cache is being invalidate because project 48dc636c17544593aaf2a2409c13e540 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:34:42.852372 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-33f002b7-64c2-470d-95fd-8026eb8435cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:42.853109 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-33f002b7-64c2-470d-95fd-8026eb8435cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:42.855826 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1257/2537] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:34:42 2022] DELETE /identity/v3/projects/48dc636c17544593aaf2a2409c13e540 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:34:42.860193 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa5d351-c1bd-4bd8-b26b-fc5bdc1e5643 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:42.861397 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa5d351-c1bd-4bd8-b26b-fc5bdc1e5643 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:42.861397 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa5d351-c1bd-4bd8-b26b-fc5bdc1e5643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:42.879551 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-efa5d351-c1bd-4bd8-b26b-fc5bdc1e5643 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:34:42.882059 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-efa5d351-c1bd-4bd8-b26b-fc5bdc1e5643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:42.927429 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f002b7-64c2-470d-95fd-8026eb8435cb admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:42.929641 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-33f002b7-64c2-470d-95fd-8026eb8435cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:42.958350 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1281/2538] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:34:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:34:42.966269 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec49e8c-d7ae-478f-aa90-956c2f6aba60 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:42.966561 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec49e8c-d7ae-478f-aa90-956c2f6aba60 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:42.966843 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec49e8c-d7ae-478f-aa90-956c2f6aba60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:42.967472 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:34:42.981895 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:34:42.988999 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-1ec49e8c-d7ae-478f-aa90-956c2f6aba60 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:34:42.990275 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-1ec49e8c-d7ae-478f-aa90-956c2f6aba60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:42.998875 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1282/2539] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:34:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:34:43.004612 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec49e8c-d7ae-478f-aa90-956c2f6aba60 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:43.014926 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec49e8c-d7ae-478f-aa90-956c2f6aba60 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:43.020147 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e81d992-b7ba-4366-8843-313d2fe4b21e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:43.020973 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e81d992-b7ba-4366-8843-313d2fe4b21e admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:43.021261 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e81d992-b7ba-4366-8843-313d2fe4b21e admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:43.021542 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e81d992-b7ba-4366-8843-313d2fe4b21e admin admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:43.021713 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f002b7-64c2-470d-95fd-8026eb8435cb admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:43.022295 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e81d992-b7ba-4366-8843-313d2fe4b21e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:34:43.022631 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e81d992-b7ba-4366-8843-313d2fe4b21e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:43.024272 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e81d992-b7ba-4366-8843-313d2fe4b21e admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:43.027595 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d00d8-7e04-4d29-9d15-4e0589fef9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:43.028567 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d00d8-7e04-4d29-9d15-4e0589fef9c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:43.028847 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d00d8-7e04-4d29-9d15-4e0589fef9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:43.029118 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d00d8-7e04-4d29-9d15-4e0589fef9c8 admin admin] PATH_INFO: `/v3/users/0f72def174a74b29b0203474c3dc2e69` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:43.030395 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d00d8-7e04-4d29-9d15-4e0589fef9c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f72def174a74b29b0203474c3dc2e69)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:43.033422 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d00d8-7e04-4d29-9d15-4e0589fef9c8 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:43.037860 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1283/2540] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:34:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:34:43.043868 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e81d992-b7ba-4366-8843-313d2fe4b21e admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:43.055670 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e81d992-b7ba-4366-8843-313d2fe4b21e admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:43.060796 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d810fde-0f16-415d-a7d9-a3c9677be023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:43.061495 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d810fde-0f16-415d-a7d9-a3c9677be023 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:43.061804 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d810fde-0f16-415d-a7d9-a3c9677be023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:43.062232 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d810fde-0f16-415d-a7d9-a3c9677be023 admin admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:43.062965 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d810fde-0f16-415d-a7d9-a3c9677be023 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:43.065718 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d810fde-0f16-415d-a7d9-a3c9677be023 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:43.275345 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1284/2541] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:34:43 2022] POST /identity/v3/projects => generated 373 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:34:43.279684 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d810fde-0f16-415d-a7d9-a3c9677be023 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:43.318525 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-2c4d00d8-7e04-4d29-9d15-4e0589fef9c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f72def174a74b29b0203474c3dc2e69'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:43.333328 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-2c4d00d8-7e04-4d29-9d15-4e0589fef9c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f72def174a74b29b0203474c3dc2e69'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:43.341483 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-2c4d00d8-7e04-4d29-9d15-4e0589fef9c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f72def174a74b29b0203474c3dc2e69'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:43.341769 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-2c4d00d8-7e04-4d29-9d15-4e0589fef9c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f72def174a74b29b0203474c3dc2e69'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:43.342554 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-2c4d00d8-7e04-4d29-9d15-4e0589fef9c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f72def174a74b29b0203474c3dc2e69'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:43.353936 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d810fde-0f16-415d-a7d9-a3c9677be023 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:43.359052 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08412af4-082d-428c-b17d-8bb173c4921a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:43.359748 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08412af4-082d-428c-b17d-8bb173c4921a admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:43.360038 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08412af4-082d-428c-b17d-8bb173c4921a admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:43.360320 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08412af4-082d-428c-b17d-8bb173c4921a admin admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:43.362064 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08412af4-082d-428c-b17d-8bb173c4921a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:43.363625 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08412af4-082d-428c-b17d-8bb173c4921a admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:43.438371 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1258/2542] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:34:42 2022] DELETE /identity/v3/users/0f72def174a74b29b0203474c3dc2e69 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:34:43.443454 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23a6f26-54ac-4fdf-8ddc-69c3440ce481 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:43.443749 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23a6f26-54ac-4fdf-8ddc-69c3440ce481 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:43.444056 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23a6f26-54ac-4fdf-8ddc-69c3440ce481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:43.458615 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1285/2543] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:34:43 2022] POST /identity/v3/users => generated 287 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:34:43.460022 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08412af4-082d-428c-b17d-8bb173c4921a admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:43.461981 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-08412af4-082d-428c-b17d-8bb173c4921a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:43.463350 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-a23a6f26-54ac-4fdf-8ddc-69c3440ce481 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:34:43.464520 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-a23a6f26-54ac-4fdf-8ddc-69c3440ce481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:43.474719 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08412af4-082d-428c-b17d-8bb173c4921a admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:43.545913 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1259/2544] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:34:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:34:43.547772 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b528b03b-eaa0-42a3-ba15-14ff73a46e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:43.548752 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b528b03b-eaa0-42a3-ba15-14ff73a46e17 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:43.549047 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b528b03b-eaa0-42a3-ba15-14ff73a46e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:43.549325 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b528b03b-eaa0-42a3-ba15-14ff73a46e17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:43.551157 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b528b03b-eaa0-42a3-ba15-14ff73a46e17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:43.552035 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b528b03b-eaa0-42a3-ba15-14ff73a46e17 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:43.557072 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23a6f26-54ac-4fdf-8ddc-69c3440ce481 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:43.558648 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-a23a6f26-54ac-4fdf-8ddc-69c3440ce481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:43.566681 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1286/2545] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:34:43 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:34:43.567865 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b528b03b-eaa0-42a3-ba15-14ff73a46e17 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:43.577994 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b528b03b-eaa0-42a3-ba15-14ff73a46e17 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:43.583213 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d547c137-5b98-4534-8679-ae0b39c7dcd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:43.583918 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d547c137-5b98-4534-8679-ae0b39c7dcd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:43.584183 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d547c137-5b98-4534-8679-ae0b39c7dcd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:43.584507 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d547c137-5b98-4534-8679-ae0b39c7dcd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:43.585413 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d547c137-5b98-4534-8679-ae0b39c7dcd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:43.587085 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d547c137-5b98-4534-8679-ae0b39c7dcd8 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:43.601224 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1287/2546] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:34:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:34:43.605745 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d547c137-5b98-4534-8679-ae0b39c7dcd8 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:43.614936 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:34:43.618337 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d547c137-5b98-4534-8679-ae0b39c7dcd8 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:43.623261 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fde088c-b7af-4918-9f2c-ad6a22de8c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:43.624168 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde088c-b7af-4918-9f2c-ad6a22de8c33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:43.624589 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde088c-b7af-4918-9f2c-ad6a22de8c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:43.624877 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde088c-b7af-4918-9f2c-ad6a22de8c33 admin admin] PATH_INFO: `/v3/projects/9a77280a811a48ab852f1359cb453e4c/users/ecb3f1339485405aaeaf374c2d73361b/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:43.630215 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23a6f26-54ac-4fdf-8ddc-69c3440ce481 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:43.639027 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53de94f7-4450-4730-8fac-975c5ea52f33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:43.639027 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53de94f7-4450-4730-8fac-975c5ea52f33 service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:43.639027 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53de94f7-4450-4730-8fac-975c5ea52f33 service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:43.639027 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53de94f7-4450-4730-8fac-975c5ea52f33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:43.641122 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fde088c-b7af-4918-9f2c-ad6a22de8c33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a77280a811a48ab852f1359cb453e4c, user_id=ecb3f1339485405aaeaf374c2d73361b, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:43.644536 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fde088c-b7af-4918-9f2c-ad6a22de8c33 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:43.646206 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53de94f7-4450-4730-8fac-975c5ea52f33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:43.648036 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53de94f7-4450-4730-8fac-975c5ea52f33 service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:43.660007 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1260/2547] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:34:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:34:43.747459 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1288/2548] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:34:43 2022] PUT /identity/v3/projects/9a77280a811a48ab852f1359cb453e4c/users/ecb3f1339485405aaeaf374c2d73361b/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:34:43.769901 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163bca89-fdf9-47fd-9b31-46021251b53f None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:43.770200 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163bca89-fdf9-47fd-9b31-46021251b53f None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:43.770667 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163bca89-fdf9-47fd-9b31-46021251b53f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:43.801680 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-163bca89-fdf9-47fd-9b31-46021251b53f None None] MFA Rules not processed for user `ecb3f1339485405aaeaf374c2d73361b`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:34:43.802800 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-163bca89-fdf9-47fd-9b31-46021251b53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:43.872649 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1261/2549] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:34:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:34:43.885892 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fde088c-b7af-4918-9f2c-ad6a22de8c33 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:43.888651 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-1fde088c-b7af-4918-9f2c-ad6a22de8c33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:43.909774 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:34:43.979118 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fde088c-b7af-4918-9f2c-ad6a22de8c33 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:43.985119 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca2b2d7-5d9a-4f24-b10b-a54a5444c354 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904255dd3c1f406eb10f6fc65f9cd098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:43.986034 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca2b2d7-5d9a-4f24-b10b-a54a5444c354 service nova] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:43.986556 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca2b2d7-5d9a-4f24-b10b-a54a5444c354 service nova] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:43.986992 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca2b2d7-5d9a-4f24-b10b-a54a5444c354 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:43.995858 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca2b2d7-5d9a-4f24-b10b-a54a5444c354 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:43.997634 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca2b2d7-5d9a-4f24-b10b-a54a5444c354 service nova] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:44.009090 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1289/2550] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:34:43 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:34:44.147371 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163bca89-fdf9-47fd-9b31-46021251b53f None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:44.149198 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-163bca89-fdf9-47fd-9b31-46021251b53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:44.234725 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163bca89-fdf9-47fd-9b31-46021251b53f None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:44.239801 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbf9c99-a37d-4718-a129-7ea551fd458c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:44.240767 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbf9c99-a37d-4718-a129-7ea551fd458c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:44.241151 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbf9c99-a37d-4718-a129-7ea551fd458c admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:44.241450 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbf9c99-a37d-4718-a129-7ea551fd458c admin admin] PATH_INFO: `/v3/projects/f4b3a84f1b0c49c0a9555ab23e096909` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:44.242713 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbf9c99-a37d-4718-a129-7ea551fd458c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4b3a84f1b0c49c0a9555ab23e096909)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:44.244718 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbf9c99-a37d-4718-a129-7ea551fd458c admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:44.358581 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-bfbf9c99-a37d-4718-a129-7ea551fd458c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4b3a84f1b0c49c0a9555ab23e096909', 'request_id': 'req-bfbf9c99-a37d-4718-a129-7ea551fd458c'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:44.466430 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-bfbf9c99-a37d-4718-a129-7ea551fd458c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4b3a84f1b0c49c0a9555ab23e096909', 'request_id': 'req-bfbf9c99-a37d-4718-a129-7ea551fd458c'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:44.473750 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-bfbf9c99-a37d-4718-a129-7ea551fd458c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4b3a84f1b0c49c0a9555ab23e096909', 'request_id': 'req-bfbf9c99-a37d-4718-a129-7ea551fd458c'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:44.490280 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-bfbf9c99-a37d-4718-a129-7ea551fd458c admin admin] The token cache is being invalidate because project f4b3a84f1b0c49c0a9555ab23e096909 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:34:44.490779 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-bfbf9c99-a37d-4718-a129-7ea551fd458c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:44.491466 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-bfbf9c99-a37d-4718-a129-7ea551fd458c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:44.493775 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1262/2551] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:34:44 2022] DELETE /identity/v3/projects/f4b3a84f1b0c49c0a9555ab23e096909 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:34:44.500343 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdfde04-ac14-44ee-9d92-2c99f2975629 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:44.500686 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdfde04-ac14-44ee-9d92-2c99f2975629 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:44.500988 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdfde04-ac14-44ee-9d92-2c99f2975629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:44.524785 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-1cdfde04-ac14-44ee-9d92-2c99f2975629 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:34:44.526068 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-1cdfde04-ac14-44ee-9d92-2c99f2975629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:44.613868 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1290/2552] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:34:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:34:44.624258 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40187990-b903-4e55-883c-e0c7ea2f7c3f None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:44.625012 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40187990-b903-4e55-883c-e0c7ea2f7c3f None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:44.625285 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40187990-b903-4e55-883c-e0c7ea2f7c3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:44.633945 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:34:44.647928 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-40187990-b903-4e55-883c-e0c7ea2f7c3f None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:34:44.650637 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-40187990-b903-4e55-883c-e0c7ea2f7c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:44.657324 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1263/2553] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:34:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:34:44.662350 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdfde04-ac14-44ee-9d92-2c99f2975629 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:44.674695 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdfde04-ac14-44ee-9d92-2c99f2975629 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:44.680175 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ef4e2e-3027-4cbe-9140-a8ed63956a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:44.680906 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ef4e2e-3027-4cbe-9140-a8ed63956a99 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:44.681192 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ef4e2e-3027-4cbe-9140-a8ed63956a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:44.681613 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ef4e2e-3027-4cbe-9140-a8ed63956a99 admin admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:44.682465 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ef4e2e-3027-4cbe-9140-a8ed63956a99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:34:44.682772 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ef4e2e-3027-4cbe-9140-a8ed63956a99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:44.685288 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ef4e2e-3027-4cbe-9140-a8ed63956a99 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:44.703026 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1291/2554] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:34:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:34:44.708139 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40187990-b903-4e55-883c-e0c7ea2f7c3f None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:44.718021 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40187990-b903-4e55-883c-e0c7ea2f7c3f None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:44.726098 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3f280d-72ba-49f0-bec5-0bad83bd3358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:44.726787 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3f280d-72ba-49f0-bec5-0bad83bd3358 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:44.727073 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3f280d-72ba-49f0-bec5-0bad83bd3358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:44.727377 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3f280d-72ba-49f0-bec5-0bad83bd3358 admin admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:44.728157 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3f280d-72ba-49f0-bec5-0bad83bd3358 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:44.730822 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3f280d-72ba-49f0-bec5-0bad83bd3358 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:44.839673 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1264/2555] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:34:44 2022] POST /identity/v3/projects => generated 381 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:34:44.843748 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ef4e2e-3027-4cbe-9140-a8ed63956a99 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:44.858966 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ef4e2e-3027-4cbe-9140-a8ed63956a99 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:44.938031 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa9fbc8-aeac-4842-a008-49680e2f621d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:44.939129 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa9fbc8-aeac-4842-a008-49680e2f621d admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:44.939419 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa9fbc8-aeac-4842-a008-49680e2f621d admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:44.939524 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa9fbc8-aeac-4842-a008-49680e2f621d admin admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:44.940225 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa9fbc8-aeac-4842-a008-49680e2f621d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:44.942195 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa9fbc8-aeac-4842-a008-49680e2f621d admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:45.047816 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1292/2556] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:34:44 2022] POST /identity/v3/users => generated 291 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:34:45.051774 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3f280d-72ba-49f0-bec5-0bad83bd3358 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:45.062512 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3f280d-72ba-49f0-bec5-0bad83bd3358 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:45.072281 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0e7de5-f246-40cc-be2a-dc7a71fdff02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:45.073992 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e7de5-f246-40cc-be2a-dc7a71fdff02 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:45.074275 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e7de5-f246-40cc-be2a-dc7a71fdff02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:45.074547 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e7de5-f246-40cc-be2a-dc7a71fdff02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:45.075292 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0e7de5-f246-40cc-be2a-dc7a71fdff02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:45.077034 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0e7de5-f246-40cc-be2a-dc7a71fdff02 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:45.090757 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1265/2557] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:34:45 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:34:45.095110 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa9fbc8-aeac-4842-a008-49680e2f621d admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:45.108340 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa9fbc8-aeac-4842-a008-49680e2f621d admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:45.118123 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763228fb-6668-4aef-95f9-efcfe70574b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:45.118917 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763228fb-6668-4aef-95f9-efcfe70574b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:45.119444 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763228fb-6668-4aef-95f9-efcfe70574b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:45.119869 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763228fb-6668-4aef-95f9-efcfe70574b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:45.120847 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763228fb-6668-4aef-95f9-efcfe70574b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:45.122751 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763228fb-6668-4aef-95f9-efcfe70574b1 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:45.136352 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1293/2558] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:34:45 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:34:45.140700 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0e7de5-f246-40cc-be2a-dc7a71fdff02 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:45.151863 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0e7de5-f246-40cc-be2a-dc7a71fdff02 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:45.156741 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b61d2b5-3746-4675-9303-5ed325945dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:45.157788 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61d2b5-3746-4675-9303-5ed325945dd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:45.158075 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61d2b5-3746-4675-9303-5ed325945dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:45.158357 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61d2b5-3746-4675-9303-5ed325945dd1 admin admin] PATH_INFO: `/v3/projects/20016956d3e2409cb41b02f7f0617daa/users/78a853ef1e51411e88ec7984693be865/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:45.176603 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b61d2b5-3746-4675-9303-5ed325945dd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20016956d3e2409cb41b02f7f0617daa, user_id=78a853ef1e51411e88ec7984693be865, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:45.179314 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b61d2b5-3746-4675-9303-5ed325945dd1 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:45.238599 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1266/2559] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:34:45 2022] PUT /identity/v3/projects/20016956d3e2409cb41b02f7f0617daa/users/78a853ef1e51411e88ec7984693be865/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:34:45.243909 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bef00f-98e7-4969-9b18-715d05a8d133 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:45.244199 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bef00f-98e7-4969-9b18-715d05a8d133 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:45.244518 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bef00f-98e7-4969-9b18-715d05a8d133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:45.259989 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:34:45.266151 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-12bef00f-98e7-4969-9b18-715d05a8d133 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:34:45.267994 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-12bef00f-98e7-4969-9b18-715d05a8d133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:45.332851 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1294/2560] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:34:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:34:45.345220 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b61d2b5-3746-4675-9303-5ed325945dd1 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:45.347396 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-4b61d2b5-3746-4675-9303-5ed325945dd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:45.411624 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b61d2b5-3746-4675-9303-5ed325945dd1 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:45.415632 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe005e8-3ed9-4c71-8be1-861306a8f992 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:45.416293 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe005e8-3ed9-4c71-8be1-861306a8f992 service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:45.416606 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe005e8-3ed9-4c71-8be1-861306a8f992 service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:45.416891 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe005e8-3ed9-4c71-8be1-861306a8f992 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:45.430448 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe005e8-3ed9-4c71-8be1-861306a8f992 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:45.431978 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe005e8-3ed9-4c71-8be1-861306a8f992 service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:45.443518 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1267/2561] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:34:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:34:46.733614 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d9eca4-ed11-48e9-bf82-1d52abeccbfa None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:46.734346 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d9eca4-ed11-48e9-bf82-1d52abeccbfa None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:46.734346 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d9eca4-ed11-48e9-bf82-1d52abeccbfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:46.768420 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-e2d9eca4-ed11-48e9-bf82-1d52abeccbfa None None] MFA Rules not processed for user `78a853ef1e51411e88ec7984693be865`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:34:46.770437 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-e2d9eca4-ed11-48e9-bf82-1d52abeccbfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:46.849152 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1295/2562] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:34:46 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:34:46.862328 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe005e8-3ed9-4c71-8be1-861306a8f992 service neutron] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:46.864663 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-cbe005e8-3ed9-4c71-8be1-861306a8f992 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:46.939560 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe005e8-3ed9-4c71-8be1-861306a8f992 service neutron] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:46.943461 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd83637-3818-40a1-adfb-02c03e91b1b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:46.944186 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd83637-3818-40a1-adfb-02c03e91b1b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:46.944515 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd83637-3818-40a1-adfb-02c03e91b1b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:46.944800 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd83637-3818-40a1-adfb-02c03e91b1b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:46.955222 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd83637-3818-40a1-adfb-02c03e91b1b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:46.956764 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd83637-3818-40a1-adfb-02c03e91b1b6 service cinder] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:46.966854 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1268/2563] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:34:46 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:34:47.416476 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d9eca4-ed11-48e9-bf82-1d52abeccbfa None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:47.416476 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-e2d9eca4-ed11-48e9-bf82-1d52abeccbfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:47.429328 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d9eca4-ed11-48e9-bf82-1d52abeccbfa None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:47.434105 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53a94d2-6cb3-42b3-80cc-23af4885b8b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:47.435000 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a94d2-6cb3-42b3-80cc-23af4885b8b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:47.435284 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a94d2-6cb3-42b3-80cc-23af4885b8b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:47.435559 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a94d2-6cb3-42b3-80cc-23af4885b8b9 admin admin] PATH_INFO: `/v3/users/ecb3f1339485405aaeaf374c2d73361b` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:47.436918 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53a94d2-6cb3-42b3-80cc-23af4885b8b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecb3f1339485405aaeaf374c2d73361b)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:47.438861 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53a94d2-6cb3-42b3-80cc-23af4885b8b9 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:47.823559 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-e53a94d2-6cb3-42b3-80cc-23af4885b8b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecb3f1339485405aaeaf374c2d73361b'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:47.837694 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-e53a94d2-6cb3-42b3-80cc-23af4885b8b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecb3f1339485405aaeaf374c2d73361b'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:47.940485 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-e53a94d2-6cb3-42b3-80cc-23af4885b8b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecb3f1339485405aaeaf374c2d73361b'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:47.941424 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-e53a94d2-6cb3-42b3-80cc-23af4885b8b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecb3f1339485405aaeaf374c2d73361b'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:47.949954 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-e53a94d2-6cb3-42b3-80cc-23af4885b8b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecb3f1339485405aaeaf374c2d73361b'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:47.956338 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1296/2564] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:34:47 2022] DELETE /identity/v3/users/ecb3f1339485405aaeaf374c2d73361b => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:34:47.961398 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea82359-250b-4e38-84ea-79bc2c595494 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:47.962019 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea82359-250b-4e38-84ea-79bc2c595494 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:47.962019 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea82359-250b-4e38-84ea-79bc2c595494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:47.979900 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-aea82359-250b-4e38-84ea-79bc2c595494 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:34:47.981186 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-aea82359-250b-4e38-84ea-79bc2c595494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:48.025994 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:34:48.045226 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1269/2565] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:34:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:34:48.055666 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53a94d2-6cb3-42b3-80cc-23af4885b8b9 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:48.057509 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-e53a94d2-6cb3-42b3-80cc-23af4885b8b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:48.069321 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:34:48.117829 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53a94d2-6cb3-42b3-80cc-23af4885b8b9 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:48.121377 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806464dd-7724-4d1a-b122-48dc46be7758 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:48.122032 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806464dd-7724-4d1a-b122-48dc46be7758 service neutron] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:48.122319 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806464dd-7724-4d1a-b122-48dc46be7758 service neutron] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:48.122617 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806464dd-7724-4d1a-b122-48dc46be7758 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:48.133332 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806464dd-7724-4d1a-b122-48dc46be7758 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:48.134769 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806464dd-7724-4d1a-b122-48dc46be7758 service neutron] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:48.145090 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1297/2566] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:34:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:34:48.703026 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea82359-250b-4e38-84ea-79bc2c595494 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:48.704548 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-aea82359-250b-4e38-84ea-79bc2c595494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:48.717783 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea82359-250b-4e38-84ea-79bc2c595494 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:48.723840 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5569595-fc0d-4fe5-bd7b-6ec28f49b7ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:48.724765 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5569595-fc0d-4fe5-bd7b-6ec28f49b7ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:48.725047 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5569595-fc0d-4fe5-bd7b-6ec28f49b7ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:48.725322 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5569595-fc0d-4fe5-bd7b-6ec28f49b7ac admin admin] PATH_INFO: `/v3/projects/9a77280a811a48ab852f1359cb453e4c` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:48.726671 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5569595-fc0d-4fe5-bd7b-6ec28f49b7ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a77280a811a48ab852f1359cb453e4c)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:48.728847 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5569595-fc0d-4fe5-bd7b-6ec28f49b7ac admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:48.862401 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-b5569595-fc0d-4fe5-bd7b-6ec28f49b7ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a77280a811a48ab852f1359cb453e4c', 'request_id': 'req-b5569595-fc0d-4fe5-bd7b-6ec28f49b7ac'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:48.909613 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c27afc5-3418-4642-906c-ef86731f10bc None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:48.910465 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c27afc5-3418-4642-906c-ef86731f10bc None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:48.917218 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c27afc5-3418-4642-906c-ef86731f10bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:48.930014 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-2c27afc5-3418-4642-906c-ef86731f10bc None None] MFA Rules not processed for user `78a853ef1e51411e88ec7984693be865`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:34:48.931341 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-2c27afc5-3418-4642-906c-ef86731f10bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:48.941095 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-b5569595-fc0d-4fe5-bd7b-6ec28f49b7ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a77280a811a48ab852f1359cb453e4c', 'request_id': 'req-b5569595-fc0d-4fe5-bd7b-6ec28f49b7ac'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:48.957588 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-b5569595-fc0d-4fe5-bd7b-6ec28f49b7ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a77280a811a48ab852f1359cb453e4c', 'request_id': 'req-b5569595-fc0d-4fe5-bd7b-6ec28f49b7ac'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:48.967186 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-b5569595-fc0d-4fe5-bd7b-6ec28f49b7ac admin admin] The token cache is being invalidate because project 9a77280a811a48ab852f1359cb453e4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:34:48.968019 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-b5569595-fc0d-4fe5-bd7b-6ec28f49b7ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:48.968205 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-b5569595-fc0d-4fe5-bd7b-6ec28f49b7ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:48.969835 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1270/2567] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:34:48 2022] DELETE /identity/v3/projects/9a77280a811a48ab852f1359cb453e4c => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:34:48.975288 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7041fcc9-0dd2-42fc-8cf2-58cbcf50cba9 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:48.975636 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7041fcc9-0dd2-42fc-8cf2-58cbcf50cba9 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:48.975923 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7041fcc9-0dd2-42fc-8cf2-58cbcf50cba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:48.995591 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-7041fcc9-0dd2-42fc-8cf2-58cbcf50cba9 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:34:48.996871 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-7041fcc9-0dd2-42fc-8cf2-58cbcf50cba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:48.997940 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1298/2568] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:34:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:34:49.073294 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1271/2569] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:34:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:34:49.083211 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbc11b2-9fce-4426-95cd-11a48a9a4061 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:49.083937 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbc11b2-9fce-4426-95cd-11a48a9a4061 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:49.084219 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbc11b2-9fce-4426-95cd-11a48a9a4061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:49.105400 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-6bbc11b2-9fce-4426-95cd-11a48a9a4061 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:34:49.106610 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-6bbc11b2-9fce-4426-95cd-11a48a9a4061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:49.115574 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1299/2570] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:34:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:34:49.119994 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7041fcc9-0dd2-42fc-8cf2-58cbcf50cba9 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:49.131001 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7041fcc9-0dd2-42fc-8cf2-58cbcf50cba9 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:49.136510 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b10d7f-faee-4c1a-a0dc-b93eb796b29d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:49.137963 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b10d7f-faee-4c1a-a0dc-b93eb796b29d admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:49.137963 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b10d7f-faee-4c1a-a0dc-b93eb796b29d admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:49.137963 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b10d7f-faee-4c1a-a0dc-b93eb796b29d admin admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:49.138655 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b10d7f-faee-4c1a-a0dc-b93eb796b29d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:34:49.138970 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b10d7f-faee-4c1a-a0dc-b93eb796b29d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:49.140749 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b10d7f-faee-4c1a-a0dc-b93eb796b29d admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:49.157511 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1272/2571] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:34:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:34:49.158982 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbc11b2-9fce-4426-95cd-11a48a9a4061 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:49.169248 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbc11b2-9fce-4426-95cd-11a48a9a4061 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:49.174145 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6eb38c-2157-4913-a8a8-f3a53c7aaa81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:49.174802 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6eb38c-2157-4913-a8a8-f3a53c7aaa81 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:49.175079 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6eb38c-2157-4913-a8a8-f3a53c7aaa81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:49.175354 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6eb38c-2157-4913-a8a8-f3a53c7aaa81 admin admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:49.176044 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6eb38c-2157-4913-a8a8-f3a53c7aaa81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:49.177975 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6eb38c-2157-4913-a8a8-f3a53c7aaa81 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:49.273913 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1300/2572] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:34:49 2022] POST /identity/v3/projects => generated 381 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:34:49.278215 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b10d7f-faee-4c1a-a0dc-b93eb796b29d admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:49.288323 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b10d7f-faee-4c1a-a0dc-b93eb796b29d admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:49.341651 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:34:49.348280 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00d9cd5-f524-44c5-9812-acb2c5b8363e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:49.348991 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d9cd5-f524-44c5-9812-acb2c5b8363e admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:49.349271 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d9cd5-f524-44c5-9812-acb2c5b8363e admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:49.350150 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d9cd5-f524-44c5-9812-acb2c5b8363e admin admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:49.350302 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00d9cd5-f524-44c5-9812-acb2c5b8363e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:49.352161 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00d9cd5-f524-44c5-9812-acb2c5b8363e admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:49.444649 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1273/2573] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:34:49 2022] POST /identity/v3/users => generated 291 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:34:49.448770 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6eb38c-2157-4913-a8a8-f3a53c7aaa81 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:49.464954 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6eb38c-2157-4913-a8a8-f3a53c7aaa81 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:49.466025 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb781ce1-035e-4384-851e-eba36ef3c172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:49.466858 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb781ce1-035e-4384-851e-eba36ef3c172 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:49.467215 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb781ce1-035e-4384-851e-eba36ef3c172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:49.467678 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb781ce1-035e-4384-851e-eba36ef3c172 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:49.468520 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb781ce1-035e-4384-851e-eba36ef3c172 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:49.470159 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb781ce1-035e-4384-851e-eba36ef3c172 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:49.487766 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1301/2574] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:34:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:34:49.488842 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00d9cd5-f524-44c5-9812-acb2c5b8363e admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:49.499041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00d9cd5-f524-44c5-9812-acb2c5b8363e admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:49.504761 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8519b42-30dc-47d9-8d29-14c5656a895e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:49.511102 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8519b42-30dc-47d9-8d29-14c5656a895e admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:49.511102 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8519b42-30dc-47d9-8d29-14c5656a895e admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:49.511102 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8519b42-30dc-47d9-8d29-14c5656a895e admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:49.511102 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8519b42-30dc-47d9-8d29-14c5656a895e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:49.511102 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8519b42-30dc-47d9-8d29-14c5656a895e admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:49.519019 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1274/2575] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:34:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:34:49.524213 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb781ce1-035e-4384-851e-eba36ef3c172 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:49.534618 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb781ce1-035e-4384-851e-eba36ef3c172 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:49.540767 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080ca62c-a127-4fc6-8872-c6834fbea483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:49.541705 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ca62c-a127-4fc6-8872-c6834fbea483 admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:49.541991 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ca62c-a127-4fc6-8872-c6834fbea483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:49.542291 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ca62c-a127-4fc6-8872-c6834fbea483 admin admin] PATH_INFO: `/v3/projects/3d79fdd1065c44d1940756025eeb6cd6/users/9a902159dc9e407ebcdc5c487b7ecb87/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:49.558637 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080ca62c-a127-4fc6-8872-c6834fbea483 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d79fdd1065c44d1940756025eeb6cd6, user_id=9a902159dc9e407ebcdc5c487b7ecb87, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:49.561384 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080ca62c-a127-4fc6-8872-c6834fbea483 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:49.644013 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1302/2576] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:34:49 2022] PUT /identity/v3/projects/3d79fdd1065c44d1940756025eeb6cd6/users/9a902159dc9e407ebcdc5c487b7ecb87/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:34:49.655791 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26de045-37da-45c7-a7e5-6e577fdec660 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:49.656087 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26de045-37da-45c7-a7e5-6e577fdec660 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:49.656370 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26de045-37da-45c7-a7e5-6e577fdec660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:49.689224 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-d26de045-37da-45c7-a7e5-6e577fdec660 None None] MFA Rules not processed for user `9a902159dc9e407ebcdc5c487b7ecb87`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:34:49.690501 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-d26de045-37da-45c7-a7e5-6e577fdec660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:49.771177 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1275/2577] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:34:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:34:49.777674 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080ca62c-a127-4fc6-8872-c6834fbea483 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:49.832948 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:34:49.843520 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080ca62c-a127-4fc6-8872-c6834fbea483 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:49.848949 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2684be76-71e7-45a7-96c8-3d60b0b25a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:49.849829 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2684be76-71e7-45a7-96c8-3d60b0b25a45 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:49.849939 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2684be76-71e7-45a7-96c8-3d60b0b25a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:49.850199 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2684be76-71e7-45a7-96c8-3d60b0b25a45 admin admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:49.850893 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2684be76-71e7-45a7-96c8-3d60b0b25a45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:49.852860 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2684be76-71e7-45a7-96c8-3d60b0b25a45 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:49.961433 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1303/2578] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:34:49 2022] POST /identity/v3/projects => generated 381 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:34:49.965516 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26de045-37da-45c7-a7e5-6e577fdec660 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:49.976703 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26de045-37da-45c7-a7e5-6e577fdec660 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:50.046002 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c1f515-ac82-41a2-8242-4775939d543f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:50.046760 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c1f515-ac82-41a2-8242-4775939d543f admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:50.047095 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c1f515-ac82-41a2-8242-4775939d543f admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:50.047345 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c1f515-ac82-41a2-8242-4775939d543f admin admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:50.048058 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c1f515-ac82-41a2-8242-4775939d543f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:50.050008 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c1f515-ac82-41a2-8242-4775939d543f admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:50.147715 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1276/2579] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:34:49 2022] POST /identity/v3/users => generated 297 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:34:50.153179 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2684be76-71e7-45a7-96c8-3d60b0b25a45 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:50.166464 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2684be76-71e7-45a7-96c8-3d60b0b25a45 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:50.171279 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819ae39e-7dfb-4ae2-ab80-4dd1122a5e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:50.171954 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819ae39e-7dfb-4ae2-ab80-4dd1122a5e2a admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:50.172235 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819ae39e-7dfb-4ae2-ab80-4dd1122a5e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:50.172547 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819ae39e-7dfb-4ae2-ab80-4dd1122a5e2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:50.173304 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819ae39e-7dfb-4ae2-ab80-4dd1122a5e2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:50.174896 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819ae39e-7dfb-4ae2-ab80-4dd1122a5e2a admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:50.187020 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1304/2580] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:34:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:34:50.191490 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c1f515-ac82-41a2-8242-4775939d543f admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:50.202417 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c1f515-ac82-41a2-8242-4775939d543f admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:50.209924 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa26dc7-2c1c-408e-82ec-d6089071722d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:50.219430 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa26dc7-2c1c-408e-82ec-d6089071722d admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:50.219430 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa26dc7-2c1c-408e-82ec-d6089071722d admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:50.219430 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa26dc7-2c1c-408e-82ec-d6089071722d admin admin] PATH_INFO: `/v3/projects/afb82c28023346a8bc40a8ebdf06e29b/users/ada09eecbb664487a81fa0368a434d54/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:50.230655 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa26dc7-2c1c-408e-82ec-d6089071722d admin admin] RBAC: Authorizing `identity:create_grant(project_id=afb82c28023346a8bc40a8ebdf06e29b, user_id=ada09eecbb664487a81fa0368a434d54, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:50.233505 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa26dc7-2c1c-408e-82ec-d6089071722d admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:50.329795 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1277/2581] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:34:50 2022] PUT /identity/v3/projects/afb82c28023346a8bc40a8ebdf06e29b/users/ada09eecbb664487a81fa0368a434d54/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:34:50.335555 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819ae39e-7dfb-4ae2-ab80-4dd1122a5e2a admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:50.354724 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819ae39e-7dfb-4ae2-ab80-4dd1122a5e2a admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:50.427002 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad938452-00fb-4477-a1e2-4e4e53f7929a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:50.427741 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad938452-00fb-4477-a1e2-4e4e53f7929a admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:50.428024 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad938452-00fb-4477-a1e2-4e4e53f7929a admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:50.428298 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad938452-00fb-4477-a1e2-4e4e53f7929a admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:50.429093 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad938452-00fb-4477-a1e2-4e4e53f7929a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:50.430772 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad938452-00fb-4477-a1e2-4e4e53f7929a admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:50.452934 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1305/2582] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:34:50 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:34:50.457727 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa26dc7-2c1c-408e-82ec-d6089071722d admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:50.474785 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa26dc7-2c1c-408e-82ec-d6089071722d admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:50.475095 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a092f7b-e347-4e19-b1f5-ba17ae7a6d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:50.476135 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a092f7b-e347-4e19-b1f5-ba17ae7a6d0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:50.476414 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a092f7b-e347-4e19-b1f5-ba17ae7a6d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:50.481953 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a092f7b-e347-4e19-b1f5-ba17ae7a6d0c admin admin] PATH_INFO: `/v3/projects/afb82c28023346a8bc40a8ebdf06e29b/users/ada09eecbb664487a81fa0368a434d54/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:50.481953 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a092f7b-e347-4e19-b1f5-ba17ae7a6d0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=afb82c28023346a8bc40a8ebdf06e29b, user_id=ada09eecbb664487a81fa0368a434d54, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:50.481953 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a092f7b-e347-4e19-b1f5-ba17ae7a6d0c admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:50.496352 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1278/2583] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:34:50 2022] PUT /identity/v3/projects/afb82c28023346a8bc40a8ebdf06e29b/users/ada09eecbb664487a81fa0368a434d54/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:34:50.507695 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8093add-5c48-4d86-8d24-e8a99c0cb724 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:50.507989 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8093add-5c48-4d86-8d24-e8a99c0cb724 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:50.508303 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8093add-5c48-4d86-8d24-e8a99c0cb724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:50.535230 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-f8093add-5c48-4d86-8d24-e8a99c0cb724 None None] MFA Rules not processed for user `ada09eecbb664487a81fa0368a434d54`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:34:50.536551 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-f8093add-5c48-4d86-8d24-e8a99c0cb724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:50.611242 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1306/2584] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:34:50 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:34:50.626034 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a092f7b-e347-4e19-b1f5-ba17ae7a6d0c admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:50.627807 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-7a092f7b-e347-4e19-b1f5-ba17ae7a6d0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:50.649464 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:34:50.693400 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a092f7b-e347-4e19-b1f5-ba17ae7a6d0c admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:50.697213 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3fe3d8-4ef0-416c-8167-d2e020f2adc0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:50.697878 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3fe3d8-4ef0-416c-8167-d2e020f2adc0 service cinder] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:50.698162 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3fe3d8-4ef0-416c-8167-d2e020f2adc0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:50.698447 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3fe3d8-4ef0-416c-8167-d2e020f2adc0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:50.710407 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3fe3d8-4ef0-416c-8167-d2e020f2adc0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:50.712083 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3fe3d8-4ef0-416c-8167-d2e020f2adc0 service cinder] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:50.723494 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1279/2585] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:34:50 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:34:51.827367 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8093add-5c48-4d86-8d24-e8a99c0cb724 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:51.837137 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8093add-5c48-4d86-8d24-e8a99c0cb724 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:51.841852 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def04c1f-da69-4db2-acae-db5a429aa380 tempest-VolumeQuotaClassesTest-1272657420 tempest-VolumeQuotaClassesTest-1272657420-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ada09eecbb664487a81fa0368a434d54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afb82c28023346a8bc40a8ebdf06e29b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:51.842589 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def04c1f-da69-4db2-acae-db5a429aa380 tempest-VolumeQuotaClassesTest-1272657420 tempest-VolumeQuotaClassesTest-1272657420-project-admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:51.846669 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def04c1f-da69-4db2-acae-db5a429aa380 tempest-VolumeQuotaClassesTest-1272657420 tempest-VolumeQuotaClassesTest-1272657420-project-admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:51.846669 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def04c1f-da69-4db2-acae-db5a429aa380 tempest-VolumeQuotaClassesTest-1272657420 tempest-VolumeQuotaClassesTest-1272657420-project-admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:51.846669 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def04c1f-da69-4db2-acae-db5a429aa380 tempest-VolumeQuotaClassesTest-1272657420 tempest-VolumeQuotaClassesTest-1272657420-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:34:51.846669 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def04c1f-da69-4db2-acae-db5a429aa380 tempest-VolumeQuotaClassesTest-1272657420 tempest-VolumeQuotaClassesTest-1272657420-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:51.846669 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def04c1f-da69-4db2-acae-db5a429aa380 tempest-VolumeQuotaClassesTest-1272657420 tempest-VolumeQuotaClassesTest-1272657420-project-admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:51.859010 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1307/2586] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:34:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:34:51.864079 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3fe3d8-4ef0-416c-8167-d2e020f2adc0 service cinder] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:51.877106 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3fe3d8-4ef0-416c-8167-d2e020f2adc0 service cinder] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:51.882445 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c86975b-cb9d-40e2-ba98-a161c43a5c5b tempest-VolumeQuotaClassesTest-1272657420 tempest-VolumeQuotaClassesTest-1272657420-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ada09eecbb664487a81fa0368a434d54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afb82c28023346a8bc40a8ebdf06e29b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:51.883288 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86975b-cb9d-40e2-ba98-a161c43a5c5b tempest-VolumeQuotaClassesTest-1272657420 tempest-VolumeQuotaClassesTest-1272657420-project-admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:51.884436 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86975b-cb9d-40e2-ba98-a161c43a5c5b tempest-VolumeQuotaClassesTest-1272657420 tempest-VolumeQuotaClassesTest-1272657420-project-admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:51.884436 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86975b-cb9d-40e2-ba98-a161c43a5c5b tempest-VolumeQuotaClassesTest-1272657420 tempest-VolumeQuotaClassesTest-1272657420-project-admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:51.884768 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c86975b-cb9d-40e2-ba98-a161c43a5c5b tempest-VolumeQuotaClassesTest-1272657420 tempest-VolumeQuotaClassesTest-1272657420-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:51.886900 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c86975b-cb9d-40e2-ba98-a161c43a5c5b tempest-VolumeQuotaClassesTest-1272657420 tempest-VolumeQuotaClassesTest-1272657420-project-admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:52.027953 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1280/2587] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:34:51 2022] POST /identity/v3/projects => generated 352 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:34:52.032197 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def04c1f-da69-4db2-acae-db5a429aa380 tempest-VolumeQuotaClassesTest-1272657420 tempest-VolumeQuotaClassesTest-1272657420-project-admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:52.043253 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def04c1f-da69-4db2-acae-db5a429aa380 tempest-VolumeQuotaClassesTest-1272657420 tempest-VolumeQuotaClassesTest-1272657420-project-admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:52.079204 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:34:52.104228 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540582c7-c108-484d-90fb-89098d0f8608 tempest-VolumeQuotaClassesTest-1272657420 tempest-VolumeQuotaClassesTest-1272657420-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ada09eecbb664487a81fa0368a434d54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afb82c28023346a8bc40a8ebdf06e29b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:52.104931 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540582c7-c108-484d-90fb-89098d0f8608 tempest-VolumeQuotaClassesTest-1272657420 tempest-VolumeQuotaClassesTest-1272657420-project-admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:52.105219 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540582c7-c108-484d-90fb-89098d0f8608 tempest-VolumeQuotaClassesTest-1272657420 tempest-VolumeQuotaClassesTest-1272657420-project-admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:52.105556 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540582c7-c108-484d-90fb-89098d0f8608 tempest-VolumeQuotaClassesTest-1272657420 tempest-VolumeQuotaClassesTest-1272657420-project-admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:52.106297 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540582c7-c108-484d-90fb-89098d0f8608 tempest-VolumeQuotaClassesTest-1272657420 tempest-VolumeQuotaClassesTest-1272657420-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:34:52.106611 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540582c7-c108-484d-90fb-89098d0f8608 tempest-VolumeQuotaClassesTest-1272657420 tempest-VolumeQuotaClassesTest-1272657420-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:52.108245 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540582c7-c108-484d-90fb-89098d0f8608 tempest-VolumeQuotaClassesTest-1272657420 tempest-VolumeQuotaClassesTest-1272657420-project-admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:52.121796 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1308/2588] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:34:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:34:52.158841 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c86975b-cb9d-40e2-ba98-a161c43a5c5b tempest-VolumeQuotaClassesTest-1272657420 tempest-VolumeQuotaClassesTest-1272657420-project-admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:52.170773 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c86975b-cb9d-40e2-ba98-a161c43a5c5b tempest-VolumeQuotaClassesTest-1272657420 tempest-VolumeQuotaClassesTest-1272657420-project-admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:52.176462 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8552830c-e7f5-4183-ac66-49624819dd7a tempest-VolumeQuotaClassesTest-1272657420 tempest-VolumeQuotaClassesTest-1272657420-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ada09eecbb664487a81fa0368a434d54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afb82c28023346a8bc40a8ebdf06e29b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:52.177355 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552830c-e7f5-4183-ac66-49624819dd7a tempest-VolumeQuotaClassesTest-1272657420 tempest-VolumeQuotaClassesTest-1272657420-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:52.177648 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552830c-e7f5-4183-ac66-49624819dd7a tempest-VolumeQuotaClassesTest-1272657420 tempest-VolumeQuotaClassesTest-1272657420-project-admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:52.177930 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552830c-e7f5-4183-ac66-49624819dd7a tempest-VolumeQuotaClassesTest-1272657420 tempest-VolumeQuotaClassesTest-1272657420-project-admin] PATH_INFO: `/v3/projects/45595944f96c43918e10b3323abb88be` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:52.179369 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8552830c-e7f5-4183-ac66-49624819dd7a tempest-VolumeQuotaClassesTest-1272657420 tempest-VolumeQuotaClassesTest-1272657420-project-admin] RBAC: Authorizing `identity:delete_project(project_id=45595944f96c43918e10b3323abb88be)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:52.181310 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8552830c-e7f5-4183-ac66-49624819dd7a tempest-VolumeQuotaClassesTest-1272657420 tempest-VolumeQuotaClassesTest-1272657420-project-admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:52.313868 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-8552830c-e7f5-4183-ac66-49624819dd7a tempest-VolumeQuotaClassesTest-1272657420 tempest-VolumeQuotaClassesTest-1272657420-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45595944f96c43918e10b3323abb88be', 'request_id': 'req-8552830c-e7f5-4183-ac66-49624819dd7a'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:52.386277 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-8552830c-e7f5-4183-ac66-49624819dd7a tempest-VolumeQuotaClassesTest-1272657420 tempest-VolumeQuotaClassesTest-1272657420-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45595944f96c43918e10b3323abb88be', 'request_id': 'req-8552830c-e7f5-4183-ac66-49624819dd7a'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:52.392596 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-8552830c-e7f5-4183-ac66-49624819dd7a tempest-VolumeQuotaClassesTest-1272657420 tempest-VolumeQuotaClassesTest-1272657420-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45595944f96c43918e10b3323abb88be', 'request_id': 'req-8552830c-e7f5-4183-ac66-49624819dd7a'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:52.405378 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-8552830c-e7f5-4183-ac66-49624819dd7a tempest-VolumeQuotaClassesTest-1272657420 tempest-VolumeQuotaClassesTest-1272657420-project-admin] The token cache is being invalidate because project 45595944f96c43918e10b3323abb88be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:34:52.405662 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-8552830c-e7f5-4183-ac66-49624819dd7a tempest-VolumeQuotaClassesTest-1272657420 tempest-VolumeQuotaClassesTest-1272657420-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:52.406332 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-8552830c-e7f5-4183-ac66-49624819dd7a tempest-VolumeQuotaClassesTest-1272657420 tempest-VolumeQuotaClassesTest-1272657420-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:52.407880 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1281/2589] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:34:52 2022] DELETE /identity/v3/projects/45595944f96c43918e10b3323abb88be => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:34:53.156532 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540582c7-c108-484d-90fb-89098d0f8608 tempest-VolumeQuotaClassesTest-1272657420 tempest-VolumeQuotaClassesTest-1272657420-project-admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:53.158332 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-540582c7-c108-484d-90fb-89098d0f8608 tempest-VolumeQuotaClassesTest-1272657420 tempest-VolumeQuotaClassesTest-1272657420-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:53.246439 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540582c7-c108-484d-90fb-89098d0f8608 tempest-VolumeQuotaClassesTest-1272657420 tempest-VolumeQuotaClassesTest-1272657420-project-admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:53.324024 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60729f9e-8f9b-4478-8f40-9db95b70d6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:53.325110 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60729f9e-8f9b-4478-8f40-9db95b70d6ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:53.325314 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60729f9e-8f9b-4478-8f40-9db95b70d6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:53.325595 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60729f9e-8f9b-4478-8f40-9db95b70d6ee admin admin] PATH_INFO: `/v3/users/9a902159dc9e407ebcdc5c487b7ecb87` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:53.327424 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60729f9e-8f9b-4478-8f40-9db95b70d6ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a902159dc9e407ebcdc5c487b7ecb87)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:53.329581 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60729f9e-8f9b-4478-8f40-9db95b70d6ee admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:53.679815 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-60729f9e-8f9b-4478-8f40-9db95b70d6ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a902159dc9e407ebcdc5c487b7ecb87'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:53.702429 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-60729f9e-8f9b-4478-8f40-9db95b70d6ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a902159dc9e407ebcdc5c487b7ecb87'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:53.809181 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-60729f9e-8f9b-4478-8f40-9db95b70d6ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a902159dc9e407ebcdc5c487b7ecb87'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:53.820926 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-60729f9e-8f9b-4478-8f40-9db95b70d6ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a902159dc9e407ebcdc5c487b7ecb87'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:53.820926 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:34:53.820926 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-60729f9e-8f9b-4478-8f40-9db95b70d6ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a902159dc9e407ebcdc5c487b7ecb87'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:53.827039 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1309/2590] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:34:53 2022] DELETE /identity/v3/users/9a902159dc9e407ebcdc5c487b7ecb87 => generated 0 bytes in 671 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:34:53.832621 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b10a8-7759-4a82-b487-542456914de7 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:53.833344 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b10a8-7759-4a82-b487-542456914de7 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:53.833344 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b10a8-7759-4a82-b487-542456914de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:53.853692 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-146b10a8-7759-4a82-b487-542456914de7 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:34:53.854847 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-146b10a8-7759-4a82-b487-542456914de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:53.928681 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1282/2591] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:34:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:34:53.945903 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60729f9e-8f9b-4478-8f40-9db95b70d6ee admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:53.947701 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-60729f9e-8f9b-4478-8f40-9db95b70d6ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:54.040091 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60729f9e-8f9b-4478-8f40-9db95b70d6ee admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:54.044637 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c207f93-e57d-4e73-bf70-db99fd126c84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:54.045335 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c207f93-e57d-4e73-bf70-db99fd126c84 service neutron] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:54.045616 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c207f93-e57d-4e73-bf70-db99fd126c84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:54.045893 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c207f93-e57d-4e73-bf70-db99fd126c84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:54.056465 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c207f93-e57d-4e73-bf70-db99fd126c84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:54.057931 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c207f93-e57d-4e73-bf70-db99fd126c84 service neutron] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:54.069948 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1310/2592] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:34:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:34:54.585008 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146b10a8-7759-4a82-b487-542456914de7 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:54.587564 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-146b10a8-7759-4a82-b487-542456914de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:54.621285 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146b10a8-7759-4a82-b487-542456914de7 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:54.628034 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b6699-7125-423f-b4c3-24e628aea8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:54.629788 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b6699-7125-423f-b4c3-24e628aea8e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:54.630113 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b6699-7125-423f-b4c3-24e628aea8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:54.630473 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b6699-7125-423f-b4c3-24e628aea8e6 admin admin] PATH_INFO: `/v3/projects/3d79fdd1065c44d1940756025eeb6cd6` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:54.640299 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871b6699-7125-423f-b4c3-24e628aea8e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d79fdd1065c44d1940756025eeb6cd6)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:54.640299 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871b6699-7125-423f-b4c3-24e628aea8e6 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:54.665100 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:34:54.801871 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-871b6699-7125-423f-b4c3-24e628aea8e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d79fdd1065c44d1940756025eeb6cd6', 'request_id': 'req-871b6699-7125-423f-b4c3-24e628aea8e6'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:54.894780 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-871b6699-7125-423f-b4c3-24e628aea8e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d79fdd1065c44d1940756025eeb6cd6', 'request_id': 'req-871b6699-7125-423f-b4c3-24e628aea8e6'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:54.906312 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-871b6699-7125-423f-b4c3-24e628aea8e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d79fdd1065c44d1940756025eeb6cd6', 'request_id': 'req-871b6699-7125-423f-b4c3-24e628aea8e6'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:54.928817 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-871b6699-7125-423f-b4c3-24e628aea8e6 admin admin] The token cache is being invalidate because project 3d79fdd1065c44d1940756025eeb6cd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:34:54.929318 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-871b6699-7125-423f-b4c3-24e628aea8e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:54.930977 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-871b6699-7125-423f-b4c3-24e628aea8e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:54.934124 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1283/2593] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:34:54 2022] DELETE /identity/v3/projects/3d79fdd1065c44d1940756025eeb6cd6 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:34:54.939264 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c207f93-e57d-4e73-bf70-db99fd126c84 service neutron] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:54.953753 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c207f93-e57d-4e73-bf70-db99fd126c84 service neutron] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:55.091511 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee10191f-1a09-4c91-8fbd-cb6216a19004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:55.093109 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee10191f-1a09-4c91-8fbd-cb6216a19004 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:55.093797 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee10191f-1a09-4c91-8fbd-cb6216a19004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:55.094302 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee10191f-1a09-4c91-8fbd-cb6216a19004 admin admin] PATH_INFO: `/v3/users/ada09eecbb664487a81fa0368a434d54` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:55.096689 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee10191f-1a09-4c91-8fbd-cb6216a19004 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ada09eecbb664487a81fa0368a434d54)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:55.101501 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee10191f-1a09-4c91-8fbd-cb6216a19004 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:55.418695 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-ee10191f-1a09-4c91-8fbd-cb6216a19004 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ada09eecbb664487a81fa0368a434d54'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:55.436183 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-ee10191f-1a09-4c91-8fbd-cb6216a19004 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ada09eecbb664487a81fa0368a434d54'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:55.524362 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-ee10191f-1a09-4c91-8fbd-cb6216a19004 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ada09eecbb664487a81fa0368a434d54'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:55.526589 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-ee10191f-1a09-4c91-8fbd-cb6216a19004 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ada09eecbb664487a81fa0368a434d54'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:55.538641 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-ee10191f-1a09-4c91-8fbd-cb6216a19004 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ada09eecbb664487a81fa0368a434d54'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:55.553070 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1311/2594] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:34:54 2022] DELETE /identity/v3/users/ada09eecbb664487a81fa0368a434d54 => generated 0 bytes in 611 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:34:56.125654 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b6699-7125-423f-b4c3-24e628aea8e6 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:56.127116 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-871b6699-7125-423f-b4c3-24e628aea8e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:56.144911 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b6699-7125-423f-b4c3-24e628aea8e6 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:56.224613 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadced75-5d52-4ef9-ac84-d61b333f6ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:56.226363 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadced75-5d52-4ef9-ac84-d61b333f6ce9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:56.227634 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadced75-5d52-4ef9-ac84-d61b333f6ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:56.228033 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadced75-5d52-4ef9-ac84-d61b333f6ce9 admin admin] PATH_INFO: `/v3/projects/afb82c28023346a8bc40a8ebdf06e29b` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:56.230661 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadced75-5d52-4ef9-ac84-d61b333f6ce9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afb82c28023346a8bc40a8ebdf06e29b)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:56.237865 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadced75-5d52-4ef9-ac84-d61b333f6ce9 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:56.379200 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-cadced75-5d52-4ef9-ac84-d61b333f6ce9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afb82c28023346a8bc40a8ebdf06e29b', 'request_id': 'req-cadced75-5d52-4ef9-ac84-d61b333f6ce9'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:56.512578 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-cadced75-5d52-4ef9-ac84-d61b333f6ce9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afb82c28023346a8bc40a8ebdf06e29b', 'request_id': 'req-cadced75-5d52-4ef9-ac84-d61b333f6ce9'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:56.524251 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-cadced75-5d52-4ef9-ac84-d61b333f6ce9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afb82c28023346a8bc40a8ebdf06e29b', 'request_id': 'req-cadced75-5d52-4ef9-ac84-d61b333f6ce9'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:56.543863 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-cadced75-5d52-4ef9-ac84-d61b333f6ce9 admin admin] The token cache is being invalidate because project afb82c28023346a8bc40a8ebdf06e29b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:34:56.544330 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-cadced75-5d52-4ef9-ac84-d61b333f6ce9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:56.546189 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-cadced75-5d52-4ef9-ac84-d61b333f6ce9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:34:56.548816 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1284/2595] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:34:56 2022] DELETE /identity/v3/projects/afb82c28023346a8bc40a8ebdf06e29b => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:34:56.554316 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c22306-680a-40c6-acb0-a8561e85de50 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:56.555018 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c22306-680a-40c6-acb0-a8561e85de50 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:56.555018 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c22306-680a-40c6-acb0-a8561e85de50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:56.579252 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-25c22306-680a-40c6-acb0-a8561e85de50 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:34:56.580519 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-25c22306-680a-40c6-acb0-a8561e85de50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:56.599703 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:34:56.659444 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1312/2596] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:34:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:34:56.668720 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef504047-e2e8-4bdd-82d5-83a06db2d649 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:56.669378 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef504047-e2e8-4bdd-82d5-83a06db2d649 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:56.670713 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef504047-e2e8-4bdd-82d5-83a06db2d649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:56.696831 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-ef504047-e2e8-4bdd-82d5-83a06db2d649 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:34:56.698089 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-ef504047-e2e8-4bdd-82d5-83a06db2d649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:56.711273 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1285/2597] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:34:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:34:56.718028 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c22306-680a-40c6-acb0-a8561e85de50 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:56.740540 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c22306-680a-40c6-acb0-a8561e85de50 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:56.751107 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce482921-ea7e-4651-beeb-47275a128a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:56.752297 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce482921-ea7e-4651-beeb-47275a128a83 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:56.752840 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce482921-ea7e-4651-beeb-47275a128a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:56.753522 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce482921-ea7e-4651-beeb-47275a128a83 admin admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:56.756115 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce482921-ea7e-4651-beeb-47275a128a83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:34:56.756732 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce482921-ea7e-4651-beeb-47275a128a83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:56.760920 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce482921-ea7e-4651-beeb-47275a128a83 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:56.796459 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1313/2598] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:34:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:34:56.798854 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef504047-e2e8-4bdd-82d5-83a06db2d649 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:56.832775 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef504047-e2e8-4bdd-82d5-83a06db2d649 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:56.868808 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8fd585-d7e2-43da-b2b2-31c34d6c0611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:56.874645 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fd585-d7e2-43da-b2b2-31c34d6c0611 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:56.875277 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fd585-d7e2-43da-b2b2-31c34d6c0611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:56.875862 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fd585-d7e2-43da-b2b2-31c34d6c0611 admin admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:56.877314 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8fd585-d7e2-43da-b2b2-31c34d6c0611 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:56.890436 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8fd585-d7e2-43da-b2b2-31c34d6c0611 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:57.001790 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1286/2599] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:34:56 2022] POST /identity/v3/projects => generated 391 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:34:57.010346 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce482921-ea7e-4651-beeb-47275a128a83 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:57.032144 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce482921-ea7e-4651-beeb-47275a128a83 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:57.116612 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b64489-f2e1-4080-93b6-81f08ddf2c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:57.117936 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b64489-f2e1-4080-93b6-81f08ddf2c4e admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:57.118491 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b64489-f2e1-4080-93b6-81f08ddf2c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:57.119020 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b64489-f2e1-4080-93b6-81f08ddf2c4e admin admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:57.120459 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b64489-f2e1-4080-93b6-81f08ddf2c4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:57.125466 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b64489-f2e1-4080-93b6-81f08ddf2c4e admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:57.225598 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1314/2600] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:34:57 2022] POST /identity/v3/users => generated 296 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:34:57.229614 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8fd585-d7e2-43da-b2b2-31c34d6c0611 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:57.237362 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:34:57.241803 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8fd585-d7e2-43da-b2b2-31c34d6c0611 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:57.247289 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b79d10-ce97-4010-af95-551dc48ed7fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:57.248114 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b79d10-ce97-4010-af95-551dc48ed7fa admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:57.252732 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b79d10-ce97-4010-af95-551dc48ed7fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:57.252732 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b79d10-ce97-4010-af95-551dc48ed7fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:57.252732 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b79d10-ce97-4010-af95-551dc48ed7fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:57.252732 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b79d10-ce97-4010-af95-551dc48ed7fa admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:57.263795 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1287/2601] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:34:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:34:57.268188 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b64489-f2e1-4080-93b6-81f08ddf2c4e admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:57.281477 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b64489-f2e1-4080-93b6-81f08ddf2c4e admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:57.285206 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc90ff00-e202-4c6a-8564-054e5693ab91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:57.286035 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc90ff00-e202-4c6a-8564-054e5693ab91 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:57.286326 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc90ff00-e202-4c6a-8564-054e5693ab91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:57.286605 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc90ff00-e202-4c6a-8564-054e5693ab91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:57.287357 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc90ff00-e202-4c6a-8564-054e5693ab91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:57.289024 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc90ff00-e202-4c6a-8564-054e5693ab91 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:57.299998 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1315/2602] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:34:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:34:57.304591 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b79d10-ce97-4010-af95-551dc48ed7fa admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:57.316018 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b79d10-ce97-4010-af95-551dc48ed7fa admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:57.321334 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2771dee-4013-4341-a17c-41e70b0c1eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:57.322240 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2771dee-4013-4341-a17c-41e70b0c1eb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:57.322521 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2771dee-4013-4341-a17c-41e70b0c1eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:57.322797 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2771dee-4013-4341-a17c-41e70b0c1eb1 admin admin] PATH_INFO: `/v3/projects/9680f595faee4991bb493c690b41006c/users/c1acdaa2cf934f4f810a748bede0fece/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:57.343537 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2771dee-4013-4341-a17c-41e70b0c1eb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9680f595faee4991bb493c690b41006c, user_id=c1acdaa2cf934f4f810a748bede0fece, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:57.346795 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2771dee-4013-4341-a17c-41e70b0c1eb1 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:57.433486 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1288/2603] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:34:57 2022] PUT /identity/v3/projects/9680f595faee4991bb493c690b41006c/users/c1acdaa2cf934f4f810a748bede0fece/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:34:57.444695 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c98b028-e7e7-4eef-ac3a-3f15f5f6c000 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:57.444988 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c98b028-e7e7-4eef-ac3a-3f15f5f6c000 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:57.445272 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c98b028-e7e7-4eef-ac3a-3f15f5f6c000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:57.477106 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-8c98b028-e7e7-4eef-ac3a-3f15f5f6c000 None None] MFA Rules not processed for user `c1acdaa2cf934f4f810a748bede0fece`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:34:57.478339 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-8c98b028-e7e7-4eef-ac3a-3f15f5f6c000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:57.554622 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1316/2604] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:34:57 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:34:57.559899 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2771dee-4013-4341-a17c-41e70b0c1eb1 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:57.657379 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2771dee-4013-4341-a17c-41e70b0c1eb1 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:57.664739 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cced27fa-fde6-4e95-9058-612d94ce82fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:57.665468 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced27fa-fde6-4e95-9058-612d94ce82fb admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:57.665759 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced27fa-fde6-4e95-9058-612d94ce82fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:57.666074 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced27fa-fde6-4e95-9058-612d94ce82fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:57.668423 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cced27fa-fde6-4e95-9058-612d94ce82fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:57.672289 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cced27fa-fde6-4e95-9058-612d94ce82fb admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:57.799846 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1289/2605] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:34:57 2022] POST /identity/v3/projects => generated 393 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:34:57.804873 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c98b028-e7e7-4eef-ac3a-3f15f5f6c000 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:57.816975 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c98b028-e7e7-4eef-ac3a-3f15f5f6c000 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:57.826660 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:34:57.877941 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd30d1b9-8625-4a3c-ba75-d3dc7a4b9099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:57.878719 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd30d1b9-8625-4a3c-ba75-d3dc7a4b9099 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:57.879199 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd30d1b9-8625-4a3c-ba75-d3dc7a4b9099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:57.879541 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd30d1b9-8625-4a3c-ba75-d3dc7a4b9099 admin admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:57.880350 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd30d1b9-8625-4a3c-ba75-d3dc7a4b9099 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:57.882450 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd30d1b9-8625-4a3c-ba75-d3dc7a4b9099 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:57.981392 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1317/2606] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:34:57 2022] POST /identity/v3/users => generated 303 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:34:57.981624 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cced27fa-fde6-4e95-9058-612d94ce82fb admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:57.993575 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cced27fa-fde6-4e95-9058-612d94ce82fb admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:58.007287 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e487bd4-3418-4938-97fe-aa263720fe48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:58.008136 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e487bd4-3418-4938-97fe-aa263720fe48 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:58.008518 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e487bd4-3418-4938-97fe-aa263720fe48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:58.008858 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e487bd4-3418-4938-97fe-aa263720fe48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:58.009727 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e487bd4-3418-4938-97fe-aa263720fe48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:58.011449 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e487bd4-3418-4938-97fe-aa263720fe48 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:58.029335 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1290/2607] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:34:57 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:34:58.030986 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd30d1b9-8625-4a3c-ba75-d3dc7a4b9099 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:58.042952 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd30d1b9-8625-4a3c-ba75-d3dc7a4b9099 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:58.048059 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb64e72-249b-4b31-b82e-ba5513aabca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:58.049016 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb64e72-249b-4b31-b82e-ba5513aabca5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:58.049326 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb64e72-249b-4b31-b82e-ba5513aabca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:58.049603 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb64e72-249b-4b31-b82e-ba5513aabca5 admin admin] PATH_INFO: `/v3/projects/4eeaa2e6015248f5bc7175f27710ae1b/users/f5907206666f4ff7b05c1c92fd0655a2/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:58.069322 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb64e72-249b-4b31-b82e-ba5513aabca5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4eeaa2e6015248f5bc7175f27710ae1b, user_id=f5907206666f4ff7b05c1c92fd0655a2, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:58.072797 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb64e72-249b-4b31-b82e-ba5513aabca5 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:58.143904 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1318/2608] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:34:58 2022] PUT /identity/v3/projects/4eeaa2e6015248f5bc7175f27710ae1b/users/f5907206666f4ff7b05c1c92fd0655a2/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:34:58.149280 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e487bd4-3418-4938-97fe-aa263720fe48 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:58.161418 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e487bd4-3418-4938-97fe-aa263720fe48 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:58.236430 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adf2a7c-3ace-4014-9cc0-d0a89aa3233e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:58.237385 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf2a7c-3ace-4014-9cc0-d0a89aa3233e admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:58.237554 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf2a7c-3ace-4014-9cc0-d0a89aa3233e admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:58.238544 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf2a7c-3ace-4014-9cc0-d0a89aa3233e admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:58.239540 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adf2a7c-3ace-4014-9cc0-d0a89aa3233e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:58.241581 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adf2a7c-3ace-4014-9cc0-d0a89aa3233e admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:58.252950 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1291/2609] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:34:58 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:34:58.258050 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb64e72-249b-4b31-b82e-ba5513aabca5 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:58.269217 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb64e72-249b-4b31-b82e-ba5513aabca5 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:58.274408 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20d2a0c-549e-43fe-a816-e083f63e6d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:58.275318 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20d2a0c-549e-43fe-a816-e083f63e6d42 admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:58.275673 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20d2a0c-549e-43fe-a816-e083f63e6d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:58.275950 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20d2a0c-549e-43fe-a816-e083f63e6d42 admin admin] PATH_INFO: `/v3/projects/4eeaa2e6015248f5bc7175f27710ae1b/users/f5907206666f4ff7b05c1c92fd0655a2/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:58.278175 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20d2a0c-549e-43fe-a816-e083f63e6d42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4eeaa2e6015248f5bc7175f27710ae1b, user_id=f5907206666f4ff7b05c1c92fd0655a2, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:58.280991 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20d2a0c-549e-43fe-a816-e083f63e6d42 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:58.292838 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1319/2610] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:34:58 2022] PUT /identity/v3/projects/4eeaa2e6015248f5bc7175f27710ae1b/users/f5907206666f4ff7b05c1c92fd0655a2/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:34:58.305742 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0a86f-5e00-47f2-9857-f1a895f23edd None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:58.306082 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0a86f-5e00-47f2-9857-f1a895f23edd None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:58.306440 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0a86f-5e00-47f2-9857-f1a895f23edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:58.337985 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-04e0a86f-5e00-47f2-9857-f1a895f23edd None None] MFA Rules not processed for user `f5907206666f4ff7b05c1c92fd0655a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:34:58.339321 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-04e0a86f-5e00-47f2-9857-f1a895f23edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:58.398161 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:34:58.453961 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1292/2611] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:34:58 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:34:58.466236 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20d2a0c-549e-43fe-a816-e083f63e6d42 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:58.469128 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-d20d2a0c-549e-43fe-a816-e083f63e6d42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:58.548802 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20d2a0c-549e-43fe-a816-e083f63e6d42 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:58.553686 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7479d7e-ece6-4bbe-b671-1b58f3370c14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:58.554359 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7479d7e-ece6-4bbe-b671-1b58f3370c14 service cinder] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:58.554643 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7479d7e-ece6-4bbe-b671-1b58f3370c14 service cinder] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:58.554986 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7479d7e-ece6-4bbe-b671-1b58f3370c14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:58.567134 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7479d7e-ece6-4bbe-b671-1b58f3370c14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:58.568660 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7479d7e-ece6-4bbe-b671-1b58f3370c14 service cinder] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:58.579896 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1320/2612] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:34:58 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:34:58.632637 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e0a86f-5e00-47f2-9857-f1a895f23edd None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:34:58.634151 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-04e0a86f-5e00-47f2-9857-f1a895f23edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:34:58.647044 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e0a86f-5e00-47f2-9857-f1a895f23edd None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:34:58.651956 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e19ad7-c867-4348-8bb3-c6a6a51b7424 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:34:58.652667 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e19ad7-c867-4348-8bb3-c6a6a51b7424 service cinder] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:34:58.652957 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e19ad7-c867-4348-8bb3-c6a6a51b7424 service cinder] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:34:58.653254 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e19ad7-c867-4348-8bb3-c6a6a51b7424 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:34:58.731493 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e19ad7-c867-4348-8bb3-c6a6a51b7424 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:34:58.733078 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e19ad7-c867-4348-8bb3-c6a6a51b7424 service cinder] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:34:58.744524 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1293/2613] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:34:58 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:35:01.863053 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7479d7e-ece6-4bbe-b671-1b58f3370c14 service cinder] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:01.863053 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-d7479d7e-ece6-4bbe-b671-1b58f3370c14 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:01.933740 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7479d7e-ece6-4bbe-b671-1b58f3370c14 service cinder] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:01.937640 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ea3d24-865e-4479-a7a0-0bf594499f58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:01.938384 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ea3d24-865e-4479-a7a0-0bf594499f58 service neutron] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:01.938679 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ea3d24-865e-4479-a7a0-0bf594499f58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:01.939058 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ea3d24-865e-4479-a7a0-0bf594499f58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:01.940838 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-a4ea3d24-865e-4479-a7a0-0bf594499f58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:01.968853 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:35:02.035159 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ea3d24-865e-4479-a7a0-0bf594499f58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:02.036703 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ea3d24-865e-4479-a7a0-0bf594499f58 service neutron] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:02.050071 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1321/2614] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:35:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:35:04.101549 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e19ad7-c867-4348-8bb3-c6a6a51b7424 service cinder] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:04.117770 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e19ad7-c867-4348-8bb3-c6a6a51b7424 service cinder] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:04.132535 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d909fb4-a60e-458c-8ab7-478d71e5db5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:04.133659 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d909fb4-a60e-458c-8ab7-478d71e5db5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:04.134092 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d909fb4-a60e-458c-8ab7-478d71e5db5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:04.134535 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d909fb4-a60e-458c-8ab7-478d71e5db5c admin admin] PATH_INFO: `/v3/users/c1acdaa2cf934f4f810a748bede0fece` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:04.136131 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d909fb4-a60e-458c-8ab7-478d71e5db5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1acdaa2cf934f4f810a748bede0fece)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:04.139064 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d909fb4-a60e-458c-8ab7-478d71e5db5c admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:04.389351 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-3d909fb4-a60e-458c-8ab7-478d71e5db5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1acdaa2cf934f4f810a748bede0fece'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:04.406932 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-3d909fb4-a60e-458c-8ab7-478d71e5db5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1acdaa2cf934f4f810a748bede0fece'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:04.407683 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-3d909fb4-a60e-458c-8ab7-478d71e5db5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1acdaa2cf934f4f810a748bede0fece'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:04.420029 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-3d909fb4-a60e-458c-8ab7-478d71e5db5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1acdaa2cf934f4f810a748bede0fece'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:04.420843 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-3d909fb4-a60e-458c-8ab7-478d71e5db5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1acdaa2cf934f4f810a748bede0fece'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:04.523828 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1294/2615] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:35:04 2022] DELETE /identity/v3/users/c1acdaa2cf934f4f810a748bede0fece => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:35:04.528590 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c179e21-8530-4b14-8140-56b7fa861b12 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:04.529143 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c179e21-8530-4b14-8140-56b7fa861b12 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:04.529143 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c179e21-8530-4b14-8140-56b7fa861b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:04.551162 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-0c179e21-8530-4b14-8140-56b7fa861b12 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:35:04.553505 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-0c179e21-8530-4b14-8140-56b7fa861b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:04.651036 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1322/2616] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:35:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:35:04.662435 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d909fb4-a60e-458c-8ab7-478d71e5db5c admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:04.666033 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-3d909fb4-a60e-458c-8ab7-478d71e5db5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:04.734833 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:35:04.753770 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d909fb4-a60e-458c-8ab7-478d71e5db5c admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:04.758859 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b72070f-ae21-4a42-8bde-e77c4ba3bc9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:04.759525 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b72070f-ae21-4a42-8bde-e77c4ba3bc9d service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:04.759806 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b72070f-ae21-4a42-8bde-e77c4ba3bc9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:04.760091 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b72070f-ae21-4a42-8bde-e77c4ba3bc9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:04.773691 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b72070f-ae21-4a42-8bde-e77c4ba3bc9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:04.775585 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b72070f-ae21-4a42-8bde-e77c4ba3bc9d service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:04.807485 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1295/2617] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:35:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:35:05.387909 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c179e21-8530-4b14-8140-56b7fa861b12 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:05.390842 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-0c179e21-8530-4b14-8140-56b7fa861b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:05.418807 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c179e21-8530-4b14-8140-56b7fa861b12 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:05.434623 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689daea0-100e-4b3e-bfc5-bdfae821942f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:05.437282 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689daea0-100e-4b3e-bfc5-bdfae821942f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:05.438048 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689daea0-100e-4b3e-bfc5-bdfae821942f admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:05.439046 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689daea0-100e-4b3e-bfc5-bdfae821942f admin admin] PATH_INFO: `/v3/projects/9680f595faee4991bb493c690b41006c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:05.442480 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689daea0-100e-4b3e-bfc5-bdfae821942f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9680f595faee4991bb493c690b41006c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:05.446985 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689daea0-100e-4b3e-bfc5-bdfae821942f admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:05.645081 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-689daea0-100e-4b3e-bfc5-bdfae821942f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9680f595faee4991bb493c690b41006c', 'request_id': 'req-689daea0-100e-4b3e-bfc5-bdfae821942f'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:05.653089 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-689daea0-100e-4b3e-bfc5-bdfae821942f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9680f595faee4991bb493c690b41006c', 'request_id': 'req-689daea0-100e-4b3e-bfc5-bdfae821942f'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:05.659547 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-689daea0-100e-4b3e-bfc5-bdfae821942f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9680f595faee4991bb493c690b41006c', 'request_id': 'req-689daea0-100e-4b3e-bfc5-bdfae821942f'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:05.841731 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-689daea0-100e-4b3e-bfc5-bdfae821942f admin admin] The token cache is being invalidate because project 9680f595faee4991bb493c690b41006c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:35:05.841731 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-689daea0-100e-4b3e-bfc5-bdfae821942f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:05.841958 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-689daea0-100e-4b3e-bfc5-bdfae821942f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:05.844166 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1323/2618] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:35:05 2022] DELETE /identity/v3/projects/9680f595faee4991bb493c690b41006c => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:35:05.848048 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b72070f-ae21-4a42-8bde-e77c4ba3bc9d service neutron] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:05.874452 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b72070f-ae21-4a42-8bde-e77c4ba3bc9d service neutron] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:06.029024 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767112e0-d464-49b6-9c49-3856b4f65993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:06.029996 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767112e0-d464-49b6-9c49-3856b4f65993 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:06.030354 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767112e0-d464-49b6-9c49-3856b4f65993 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:06.030701 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767112e0-d464-49b6-9c49-3856b4f65993 admin admin] PATH_INFO: `/v3/users/f5907206666f4ff7b05c1c92fd0655a2` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:06.032179 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767112e0-d464-49b6-9c49-3856b4f65993 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5907206666f4ff7b05c1c92fd0655a2)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:06.034243 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767112e0-d464-49b6-9c49-3856b4f65993 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:06.323009 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-767112e0-d464-49b6-9c49-3856b4f65993 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5907206666f4ff7b05c1c92fd0655a2'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:06.346463 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-767112e0-d464-49b6-9c49-3856b4f65993 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5907206666f4ff7b05c1c92fd0655a2'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:06.348272 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-767112e0-d464-49b6-9c49-3856b4f65993 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5907206666f4ff7b05c1c92fd0655a2'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:06.371165 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-767112e0-d464-49b6-9c49-3856b4f65993 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5907206666f4ff7b05c1c92fd0655a2'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:06.371986 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-767112e0-d464-49b6-9c49-3856b4f65993 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5907206666f4ff7b05c1c92fd0655a2'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:06.492998 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1296/2619] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:35:05 2022] DELETE /identity/v3/users/f5907206666f4ff7b05c1c92fd0655a2 => generated 0 bytes in 645 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:35:06.970630 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689daea0-100e-4b3e-bfc5-bdfae821942f admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:06.973649 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-689daea0-100e-4b3e-bfc5-bdfae821942f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:07.002997 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689daea0-100e-4b3e-bfc5-bdfae821942f admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:07.065469 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:35:07.084169 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abe40b1-dab1-4a5d-b9ea-5d8eb767729f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:07.085570 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe40b1-dab1-4a5d-b9ea-5d8eb767729f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:07.085570 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe40b1-dab1-4a5d-b9ea-5d8eb767729f admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:07.085917 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe40b1-dab1-4a5d-b9ea-5d8eb767729f admin admin] PATH_INFO: `/v3/users/e03c3a73166b4bbe9fca6d18ecf59221` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:07.086923 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abe40b1-dab1-4a5d-b9ea-5d8eb767729f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e03c3a73166b4bbe9fca6d18ecf59221)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:07.088976 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abe40b1-dab1-4a5d-b9ea-5d8eb767729f admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:07.176910 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767112e0-d464-49b6-9c49-3856b4f65993 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:07.179005 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-767112e0-d464-49b6-9c49-3856b4f65993 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:07.195572 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767112e0-d464-49b6-9c49-3856b4f65993 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:07.206119 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b26fb1-f9d8-455b-a671-a68e0c58dc03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:07.207109 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b26fb1-f9d8-455b-a671-a68e0c58dc03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:07.207109 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b26fb1-f9d8-455b-a671-a68e0c58dc03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:07.207416 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b26fb1-f9d8-455b-a671-a68e0c58dc03 admin admin] PATH_INFO: `/v3/projects/4eeaa2e6015248f5bc7175f27710ae1b` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:07.209357 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b26fb1-f9d8-455b-a671-a68e0c58dc03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4eeaa2e6015248f5bc7175f27710ae1b)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:07.211330 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b26fb1-f9d8-455b-a671-a68e0c58dc03 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:07.321587 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-9abe40b1-dab1-4a5d-b9ea-5d8eb767729f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e03c3a73166b4bbe9fca6d18ecf59221'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:07.339591 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-9abe40b1-dab1-4a5d-b9ea-5d8eb767729f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e03c3a73166b4bbe9fca6d18ecf59221'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:07.364247 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-f4b26fb1-f9d8-455b-a671-a68e0c58dc03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4eeaa2e6015248f5bc7175f27710ae1b', 'request_id': 'req-f4b26fb1-f9d8-455b-a671-a68e0c58dc03'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:07.369135 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:35:07.517374 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-9abe40b1-dab1-4a5d-b9ea-5d8eb767729f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e03c3a73166b4bbe9fca6d18ecf59221'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:07.518020 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-9abe40b1-dab1-4a5d-b9ea-5d8eb767729f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e03c3a73166b4bbe9fca6d18ecf59221'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:07.529503 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-9abe40b1-dab1-4a5d-b9ea-5d8eb767729f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e03c3a73166b4bbe9fca6d18ecf59221'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:07.537382 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1324/2620] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:35:06 2022] DELETE /identity/v3/users/e03c3a73166b4bbe9fca6d18ecf59221 => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:35:07.648326 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-f4b26fb1-f9d8-455b-a671-a68e0c58dc03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4eeaa2e6015248f5bc7175f27710ae1b', 'request_id': 'req-f4b26fb1-f9d8-455b-a671-a68e0c58dc03'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:07.655716 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-f4b26fb1-f9d8-455b-a671-a68e0c58dc03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4eeaa2e6015248f5bc7175f27710ae1b', 'request_id': 'req-f4b26fb1-f9d8-455b-a671-a68e0c58dc03'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:07.670938 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-f4b26fb1-f9d8-455b-a671-a68e0c58dc03 admin admin] The token cache is being invalidate because project 4eeaa2e6015248f5bc7175f27710ae1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:35:07.671426 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-f4b26fb1-f9d8-455b-a671-a68e0c58dc03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:07.672508 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-f4b26fb1-f9d8-455b-a671-a68e0c58dc03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:07.674718 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1297/2621] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:35:07 2022] DELETE /identity/v3/projects/4eeaa2e6015248f5bc7175f27710ae1b => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:35:07.682005 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f6c9ec-f3e4-47f3-88f4-ece7d401ea67 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:07.682305 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f6c9ec-f3e4-47f3-88f4-ece7d401ea67 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:07.682703 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f6c9ec-f3e4-47f3-88f4-ece7d401ea67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:07.702902 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-38f6c9ec-f3e4-47f3-88f4-ece7d401ea67 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:35:07.704247 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-38f6c9ec-f3e4-47f3-88f4-ece7d401ea67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:07.778584 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1325/2622] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:35:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:35:07.788852 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def2d3db-4009-45ad-8b21-d309766f1bfd None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:07.789190 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def2d3db-4009-45ad-8b21-d309766f1bfd None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:07.789619 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def2d3db-4009-45ad-8b21-d309766f1bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:07.811328 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-def2d3db-4009-45ad-8b21-d309766f1bfd None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:35:07.812772 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-def2d3db-4009-45ad-8b21-d309766f1bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:07.821371 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1298/2623] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:35:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:35:07.825984 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f6c9ec-f3e4-47f3-88f4-ece7d401ea67 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:07.840707 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f6c9ec-f3e4-47f3-88f4-ece7d401ea67 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:07.841643 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def2d3db-4009-45ad-8b21-d309766f1bfd None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:07.843551 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-def2d3db-4009-45ad-8b21-d309766f1bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:07.853895 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b15f6c-b387-4878-aca1-c24e0e07ec74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:07.853895 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b15f6c-b387-4878-aca1-c24e0e07ec74 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:07.859103 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b15f6c-b387-4878-aca1-c24e0e07ec74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:07.859365 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b15f6c-b387-4878-aca1-c24e0e07ec74 admin admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:07.860180 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b15f6c-b387-4878-aca1-c24e0e07ec74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:35:07.860557 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b15f6c-b387-4878-aca1-c24e0e07ec74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:07.862856 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b15f6c-b387-4878-aca1-c24e0e07ec74 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:07.868034 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def2d3db-4009-45ad-8b21-d309766f1bfd None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:07.876596 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0e9393-8f48-4b88-89f9-4735491cea59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:07.877805 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e9393-8f48-4b88-89f9-4735491cea59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:07.884819 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e9393-8f48-4b88-89f9-4735491cea59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:07.885120 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1326/2624] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:35:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:35:07.885309 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e9393-8f48-4b88-89f9-4735491cea59 admin admin] PATH_INFO: `/v3/projects/227bb7e0aca249aea1d7bf2ecb2903e9` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:07.885309 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0e9393-8f48-4b88-89f9-4735491cea59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=227bb7e0aca249aea1d7bf2ecb2903e9)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:07.885904 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b15f6c-b387-4878-aca1-c24e0e07ec74 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:07.891210 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0e9393-8f48-4b88-89f9-4735491cea59 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:07.898487 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b15f6c-b387-4878-aca1-c24e0e07ec74 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:07.904472 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8275d13e-b96e-4b10-848c-dd35de1fa664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:07.905147 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8275d13e-b96e-4b10-848c-dd35de1fa664 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:07.905437 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8275d13e-b96e-4b10-848c-dd35de1fa664 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:07.905744 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8275d13e-b96e-4b10-848c-dd35de1fa664 admin admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:07.906452 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8275d13e-b96e-4b10-848c-dd35de1fa664 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:07.908505 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8275d13e-b96e-4b10-848c-dd35de1fa664 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:08.062944 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1327/2625] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:35:07 2022] POST /identity/v3/projects => generated 377 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:35:08.067563 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8275d13e-b96e-4b10-848c-dd35de1fa664 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:08.107466 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-2c0e9393-8f48-4b88-89f9-4735491cea59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '227bb7e0aca249aea1d7bf2ecb2903e9', 'request_id': 'req-2c0e9393-8f48-4b88-89f9-4735491cea59'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:08.163381 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8275d13e-b96e-4b10-848c-dd35de1fa664 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:08.191996 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-2c0e9393-8f48-4b88-89f9-4735491cea59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '227bb7e0aca249aea1d7bf2ecb2903e9', 'request_id': 'req-2c0e9393-8f48-4b88-89f9-4735491cea59'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:08.202450 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-2c0e9393-8f48-4b88-89f9-4735491cea59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '227bb7e0aca249aea1d7bf2ecb2903e9', 'request_id': 'req-2c0e9393-8f48-4b88-89f9-4735491cea59'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:08.222665 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:35:08.227793 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-2c0e9393-8f48-4b88-89f9-4735491cea59 admin admin] The token cache is being invalidate because project 227bb7e0aca249aea1d7bf2ecb2903e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:35:08.233073 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-2c0e9393-8f48-4b88-89f9-4735491cea59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:08.233073 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-2c0e9393-8f48-4b88-89f9-4735491cea59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:08.234190 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1299/2626] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:35:07 2022] DELETE /identity/v3/projects/227bb7e0aca249aea1d7bf2ecb2903e9 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:35:08.242676 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c4560-a807-482d-abc2-e21e303d23be None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:08.243185 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c4560-a807-482d-abc2-e21e303d23be None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:08.243701 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c4560-a807-482d-abc2-e21e303d23be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:08.256650 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2156110-8535-40d3-9632-a902596e4ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:08.257548 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2156110-8535-40d3-9632-a902596e4ee3 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:08.257945 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2156110-8535-40d3-9632-a902596e4ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:08.258260 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2156110-8535-40d3-9632-a902596e4ee3 admin admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:08.259773 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2156110-8535-40d3-9632-a902596e4ee3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:08.264538 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2156110-8535-40d3-9632-a902596e4ee3 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:08.278095 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-fe5c4560-a807-482d-abc2-e21e303d23be None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:35:08.279542 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-fe5c4560-a807-482d-abc2-e21e303d23be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:08.292589 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1300/2627] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:35:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:35:08.304004 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6c1d1-e30f-419b-8abf-e03730965887 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:08.304519 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6c1d1-e30f-419b-8abf-e03730965887 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:08.305076 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6c1d1-e30f-419b-8abf-e03730965887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:08.328632 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-84a6c1d1-e30f-419b-8abf-e03730965887 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:35:08.329911 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-84a6c1d1-e30f-419b-8abf-e03730965887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:08.339716 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1301/2628] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:35:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:35:08.346044 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a6c1d1-e30f-419b-8abf-e03730965887 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:08.359702 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a6c1d1-e30f-419b-8abf-e03730965887 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:08.365544 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50a72c9-3692-40cf-866a-6e74fc2928f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:08.366248 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a72c9-3692-40cf-866a-6e74fc2928f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:08.366618 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a72c9-3692-40cf-866a-6e74fc2928f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:08.366852 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a72c9-3692-40cf-866a-6e74fc2928f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:08.367610 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50a72c9-3692-40cf-866a-6e74fc2928f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:35:08.367936 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50a72c9-3692-40cf-866a-6e74fc2928f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:08.369657 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50a72c9-3692-40cf-866a-6e74fc2928f5 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:08.393164 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1328/2629] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:35:08 2022] POST /identity/v3/users => generated 289 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:35:08.393410 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1302/2630] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:35:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:35:08.395429 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2156110-8535-40d3-9632-a902596e4ee3 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:08.396705 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50a72c9-3692-40cf-866a-6e74fc2928f5 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:08.400031 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-f2156110-8535-40d3-9632-a902596e4ee3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:08.408547 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50a72c9-3692-40cf-866a-6e74fc2928f5 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:08.414358 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114f0789-dfb9-4371-a2f6-df8ff7284ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:08.415066 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f0789-dfb9-4371-a2f6-df8ff7284ed5 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:08.415350 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f0789-dfb9-4371-a2f6-df8ff7284ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:08.415623 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f0789-dfb9-4371-a2f6-df8ff7284ed5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:08.416329 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114f0789-dfb9-4371-a2f6-df8ff7284ed5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:08.418487 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114f0789-dfb9-4371-a2f6-df8ff7284ed5 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:08.423983 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2156110-8535-40d3-9632-a902596e4ee3 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:08.433314 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94685697-d361-4fc8-8e89-d75266b7076a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:08.434471 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94685697-d361-4fc8-8e89-d75266b7076a admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:08.434836 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94685697-d361-4fc8-8e89-d75266b7076a admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:08.435511 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94685697-d361-4fc8-8e89-d75266b7076a admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:08.438648 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94685697-d361-4fc8-8e89-d75266b7076a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:08.439758 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94685697-d361-4fc8-8e89-d75266b7076a admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:08.451856 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1329/2631] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:35:08 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:35:08.461230 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94685697-d361-4fc8-8e89-d75266b7076a admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:08.473174 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94685697-d361-4fc8-8e89-d75266b7076a admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:08.481500 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4520ce65-cf93-4d29-a6ce-2e01ba77dcce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:08.482295 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520ce65-cf93-4d29-a6ce-2e01ba77dcce admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:08.482649 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520ce65-cf93-4d29-a6ce-2e01ba77dcce admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:08.483190 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520ce65-cf93-4d29-a6ce-2e01ba77dcce admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:08.484001 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4520ce65-cf93-4d29-a6ce-2e01ba77dcce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:08.485786 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4520ce65-cf93-4d29-a6ce-2e01ba77dcce admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:08.499437 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1330/2632] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:35:08 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:35:08.504495 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4520ce65-cf93-4d29-a6ce-2e01ba77dcce admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:08.515927 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4520ce65-cf93-4d29-a6ce-2e01ba77dcce admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:08.522376 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f16270-1785-4d42-af10-4258d7ec0764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:08.522840 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f16270-1785-4d42-af10-4258d7ec0764 admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:08.523125 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f16270-1785-4d42-af10-4258d7ec0764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:08.523400 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f16270-1785-4d42-af10-4258d7ec0764 admin admin] PATH_INFO: `/v3/projects/bbda57dfe03745d5afa4b04bf6e568e6/users/07bbe73b83814657b2a46e989a500f93/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:08.542247 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f16270-1785-4d42-af10-4258d7ec0764 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbda57dfe03745d5afa4b04bf6e568e6, user_id=07bbe73b83814657b2a46e989a500f93, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:08.545637 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f16270-1785-4d42-af10-4258d7ec0764 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:08.572852 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1303/2633] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:35:08 2022] POST /identity/v3/projects => generated 377 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:35:08.578472 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114f0789-dfb9-4371-a2f6-df8ff7284ed5 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:08.661106 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1331/2634] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:35:08 2022] PUT /identity/v3/projects/bbda57dfe03745d5afa4b04bf6e568e6/users/07bbe73b83814657b2a46e989a500f93/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:35:08.662940 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f16270-1785-4d42-af10-4258d7ec0764 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:08.665578 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-70f16270-1785-4d42-af10-4258d7ec0764 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:08.688754 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114f0789-dfb9-4371-a2f6-df8ff7284ed5 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:08.702905 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:35:08.763584 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7e5752-a00e-42de-b6f0-5e32edad2ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:08.764589 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e5752-a00e-42de-b6f0-5e32edad2ebc admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:08.764589 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e5752-a00e-42de-b6f0-5e32edad2ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:08.764879 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e5752-a00e-42de-b6f0-5e32edad2ebc admin admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:08.765606 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7e5752-a00e-42de-b6f0-5e32edad2ebc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:08.767576 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7e5752-a00e-42de-b6f0-5e32edad2ebc admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:08.796853 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f16270-1785-4d42-af10-4258d7ec0764 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:08.802925 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af75ffbd-e2e8-4875-936b-5d1aab9300c9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938187967efe49be9c5389d1feef81ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:08.803609 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75ffbd-e2e8-4875-936b-5d1aab9300c9 service placement] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:08.803904 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75ffbd-e2e8-4875-936b-5d1aab9300c9 service placement] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:08.804188 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75ffbd-e2e8-4875-936b-5d1aab9300c9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:08.806318 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-af75ffbd-e2e8-4875-936b-5d1aab9300c9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:08.821408 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af75ffbd-e2e8-4875-936b-5d1aab9300c9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:08.823020 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af75ffbd-e2e8-4875-936b-5d1aab9300c9 service placement] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:08.835310 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1332/2635] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:35:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:35:08.836686 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620bcc0e-6b50-4a7d-95a1-f21775fb56a9 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:08.836976 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620bcc0e-6b50-4a7d-95a1-f21775fb56a9 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:08.837251 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620bcc0e-6b50-4a7d-95a1-f21775fb56a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:08.851383 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:35:08.870299 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-620bcc0e-6b50-4a7d-95a1-f21775fb56a9 None None] MFA Rules not processed for user `07bbe73b83814657b2a46e989a500f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:35:08.871586 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-620bcc0e-6b50-4a7d-95a1-f21775fb56a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:08.914124 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1304/2636] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:35:08 2022] POST /identity/v3/users => generated 289 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:35:08.919702 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7e5752-a00e-42de-b6f0-5e32edad2ebc admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:08.936241 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7e5752-a00e-42de-b6f0-5e32edad2ebc admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:08.941792 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3868a6-1c25-45d0-8c7e-f9cfe267f658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:08.942514 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3868a6-1c25-45d0-8c7e-f9cfe267f658 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:08.943350 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3868a6-1c25-45d0-8c7e-f9cfe267f658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:08.943668 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3868a6-1c25-45d0-8c7e-f9cfe267f658 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:08.944476 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3868a6-1c25-45d0-8c7e-f9cfe267f658 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:08.946147 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3868a6-1c25-45d0-8c7e-f9cfe267f658 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:08.959076 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1305/2637] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:35:08 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:35:08.968334 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3868a6-1c25-45d0-8c7e-f9cfe267f658 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:08.987288 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1333/2638] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:35:08 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:35:08.987680 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3868a6-1c25-45d0-8c7e-f9cfe267f658 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:08.995982 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa160e9-5585-4381-a40d-16027b8f274b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:08.996898 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa160e9-5585-4381-a40d-16027b8f274b admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:08.997438 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa160e9-5585-4381-a40d-16027b8f274b admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:08.997964 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa160e9-5585-4381-a40d-16027b8f274b admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:08.999469 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa160e9-5585-4381-a40d-16027b8f274b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:09.001213 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa160e9-5585-4381-a40d-16027b8f274b admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:09.002077 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620bcc0e-6b50-4a7d-95a1-f21775fb56a9 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:09.004755 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-620bcc0e-6b50-4a7d-95a1-f21775fb56a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:09.017332 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1306/2639] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:35:08 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:35:09.022714 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa160e9-5585-4381-a40d-16027b8f274b admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:09.042960 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa160e9-5585-4381-a40d-16027b8f274b admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:09.084982 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d31c0c-8d21-4433-80a9-79ae5302fa2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:09.084982 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d31c0c-8d21-4433-80a9-79ae5302fa2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:09.084982 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d31c0c-8d21-4433-80a9-79ae5302fa2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:09.084982 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d31c0c-8d21-4433-80a9-79ae5302fa2a admin admin] PATH_INFO: `/v3/projects/466c7ed97ff34cd7824164207ab764d0/users/5b424248bfb84fe4a25269232e08dcaf/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:09.086479 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d31c0c-8d21-4433-80a9-79ae5302fa2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=466c7ed97ff34cd7824164207ab764d0, user_id=5b424248bfb84fe4a25269232e08dcaf, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:09.086479 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d31c0c-8d21-4433-80a9-79ae5302fa2a admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:09.094303 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620bcc0e-6b50-4a7d-95a1-f21775fb56a9 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:09.112899 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08cc164-f26c-4374-a981-9177259c0780 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9672794bc1ca4dcd8259d51d9290384b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:09.112899 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08cc164-f26c-4374-a981-9177259c0780 service glance] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:09.112899 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08cc164-f26c-4374-a981-9177259c0780 service glance] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:09.112899 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08cc164-f26c-4374-a981-9177259c0780 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:09.114041 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08cc164-f26c-4374-a981-9177259c0780 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:09.114041 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08cc164-f26c-4374-a981-9177259c0780 service glance] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:09.124456 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1334/2640] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:35:09 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:35:09.305838 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1307/2641] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:35:09 2022] PUT /identity/v3/projects/466c7ed97ff34cd7824164207ab764d0/users/5b424248bfb84fe4a25269232e08dcaf/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:35:09.313967 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c26f69c-8477-44c6-8945-9ae5129f15a0 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:09.314477 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c26f69c-8477-44c6-8945-9ae5129f15a0 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:09.314970 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c26f69c-8477-44c6-8945-9ae5129f15a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:09.346962 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-6c26f69c-8477-44c6-8945-9ae5129f15a0 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:35:09.349817 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-6c26f69c-8477-44c6-8945-9ae5129f15a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:09.435745 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1335/2642] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:35:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:35:09.452818 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d31c0c-8d21-4433-80a9-79ae5302fa2a admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:09.454639 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-a0d31c0c-8d21-4433-80a9-79ae5302fa2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:09.534903 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d31c0c-8d21-4433-80a9-79ae5302fa2a admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:09.539563 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce47825f-808f-4176-9f07-39c772ebecf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:09.540244 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce47825f-808f-4176-9f07-39c772ebecf2 service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:09.540572 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce47825f-808f-4176-9f07-39c772ebecf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:09.540864 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce47825f-808f-4176-9f07-39c772ebecf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:09.551665 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce47825f-808f-4176-9f07-39c772ebecf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:09.554580 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce47825f-808f-4176-9f07-39c772ebecf2 service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:09.566291 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1308/2643] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:35:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:35:16.947248 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8290fe-2400-4bac-a64d-ea4c6a292d20 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:16.948061 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8290fe-2400-4bac-a64d-ea4c6a292d20 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:16.948061 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8290fe-2400-4bac-a64d-ea4c6a292d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:16.998988 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-4c8290fe-2400-4bac-a64d-ea4c6a292d20 None None] MFA Rules not processed for user `5b424248bfb84fe4a25269232e08dcaf`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:35:17.001079 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-4c8290fe-2400-4bac-a64d-ea4c6a292d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:17.043896 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:35:17.098809 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1336/2644] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:35:16 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:35:17.115617 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce47825f-808f-4176-9f07-39c772ebecf2 service neutron] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:17.117317 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-ce47825f-808f-4176-9f07-39c772ebecf2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:17.205510 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:35:17.236215 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce47825f-808f-4176-9f07-39c772ebecf2 service neutron] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:17.248362 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd776342-8884-40f6-bb1b-b39b34cd5748 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904255dd3c1f406eb10f6fc65f9cd098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:17.248754 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd776342-8884-40f6-bb1b-b39b34cd5748 service nova] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:17.248971 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd776342-8884-40f6-bb1b-b39b34cd5748 service nova] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:17.249444 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd776342-8884-40f6-bb1b-b39b34cd5748 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:17.267553 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd776342-8884-40f6-bb1b-b39b34cd5748 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:17.272178 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd776342-8884-40f6-bb1b-b39b34cd5748 service nova] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:17.304558 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1309/2645] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:35:17 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:35:21.625580 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae0007-a6a3-455f-b723-d938767081fa None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:21.626215 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae0007-a6a3-455f-b723-d938767081fa None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:21.626215 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae0007-a6a3-455f-b723-d938767081fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:21.649949 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-20ae0007-a6a3-455f-b723-d938767081fa None None] MFA Rules not processed for user `5b424248bfb84fe4a25269232e08dcaf`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:35:21.651224 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-20ae0007-a6a3-455f-b723-d938767081fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:21.658776 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1337/2646] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:35:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:35:27.340559 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=6 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:35:29.417068 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd776342-8884-40f6-bb1b-b39b34cd5748 service nova] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:29.430931 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd776342-8884-40f6-bb1b-b39b34cd5748 service nova] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:29.436915 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21878ae1-fa71-4bb6-9124-b46ba80c4d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:29.438376 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21878ae1-fa71-4bb6-9124-b46ba80c4d4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:29.438702 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21878ae1-fa71-4bb6-9124-b46ba80c4d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:29.439034 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21878ae1-fa71-4bb6-9124-b46ba80c4d4e admin admin] PATH_INFO: `/v3/users/07bbe73b83814657b2a46e989a500f93` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:29.441012 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21878ae1-fa71-4bb6-9124-b46ba80c4d4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=07bbe73b83814657b2a46e989a500f93)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:29.445459 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21878ae1-fa71-4bb6-9124-b46ba80c4d4e admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:29.747877 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-21878ae1-fa71-4bb6-9124-b46ba80c4d4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07bbe73b83814657b2a46e989a500f93'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:29.767429 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-21878ae1-fa71-4bb6-9124-b46ba80c4d4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07bbe73b83814657b2a46e989a500f93'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:29.768195 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-21878ae1-fa71-4bb6-9124-b46ba80c4d4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07bbe73b83814657b2a46e989a500f93'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:29.778619 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-21878ae1-fa71-4bb6-9124-b46ba80c4d4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07bbe73b83814657b2a46e989a500f93'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:29.779387 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-21878ae1-fa71-4bb6-9124-b46ba80c4d4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07bbe73b83814657b2a46e989a500f93'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:29.895183 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1310/2647] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:35:29 2022] DELETE /identity/v3/users/07bbe73b83814657b2a46e989a500f93 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:35:29.901340 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e329ab0-89a3-4445-989b-61b74dcfae36 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:29.901990 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e329ab0-89a3-4445-989b-61b74dcfae36 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:29.902219 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e329ab0-89a3-4445-989b-61b74dcfae36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:29.930438 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-6e329ab0-89a3-4445-989b-61b74dcfae36 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:35:29.932574 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-6e329ab0-89a3-4445-989b-61b74dcfae36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:30.007026 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1338/2648] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:35:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:35:30.021007 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21878ae1-fa71-4bb6-9124-b46ba80c4d4e admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:30.023168 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-21878ae1-fa71-4bb6-9124-b46ba80c4d4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:30.094376 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21878ae1-fa71-4bb6-9124-b46ba80c4d4e admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:30.098208 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dece864-4211-48e8-afc8-a5b339bd99d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:30.099393 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dece864-4211-48e8-afc8-a5b339bd99d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:30.099683 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dece864-4211-48e8-afc8-a5b339bd99d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:30.099965 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dece864-4211-48e8-afc8-a5b339bd99d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:30.110703 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dece864-4211-48e8-afc8-a5b339bd99d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:30.112178 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dece864-4211-48e8-afc8-a5b339bd99d3 service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:30.125120 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1311/2649] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:35:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:35:30.601027 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e329ab0-89a3-4445-989b-61b74dcfae36 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:30.603452 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-6e329ab0-89a3-4445-989b-61b74dcfae36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:30.626076 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e329ab0-89a3-4445-989b-61b74dcfae36 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:30.635386 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e990714-f807-414e-82ba-a3d6fe9e3369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:30.636752 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e990714-f807-414e-82ba-a3d6fe9e3369 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:30.647499 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e990714-f807-414e-82ba-a3d6fe9e3369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:30.647499 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e990714-f807-414e-82ba-a3d6fe9e3369 admin admin] PATH_INFO: `/v3/projects/bbda57dfe03745d5afa4b04bf6e568e6` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:30.647499 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e990714-f807-414e-82ba-a3d6fe9e3369 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbda57dfe03745d5afa4b04bf6e568e6)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:30.647499 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e990714-f807-414e-82ba-a3d6fe9e3369 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:30.840026 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-5e990714-f807-414e-82ba-a3d6fe9e3369 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbda57dfe03745d5afa4b04bf6e568e6', 'request_id': 'req-5e990714-f807-414e-82ba-a3d6fe9e3369'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:30.852022 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-5e990714-f807-414e-82ba-a3d6fe9e3369 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbda57dfe03745d5afa4b04bf6e568e6', 'request_id': 'req-5e990714-f807-414e-82ba-a3d6fe9e3369'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:30.859974 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-5e990714-f807-414e-82ba-a3d6fe9e3369 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbda57dfe03745d5afa4b04bf6e568e6', 'request_id': 'req-5e990714-f807-414e-82ba-a3d6fe9e3369'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:30.999029 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-5e990714-f807-414e-82ba-a3d6fe9e3369 admin admin] The token cache is being invalidate because project bbda57dfe03745d5afa4b04bf6e568e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:35:30.999817 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-5e990714-f807-414e-82ba-a3d6fe9e3369 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:31.000170 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-5e990714-f807-414e-82ba-a3d6fe9e3369 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:31.001873 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1339/2650] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:35:30 2022] DELETE /identity/v3/projects/bbda57dfe03745d5afa4b04bf6e568e6 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:35:31.008244 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e16b2-a594-48e7-b1cb-9a614cc7cedb None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:31.020019 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e16b2-a594-48e7-b1cb-9a614cc7cedb None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:31.020019 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e16b2-a594-48e7-b1cb-9a614cc7cedb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:31.029675 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-e10e16b2-a594-48e7-b1cb-9a614cc7cedb None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:35:31.030933 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-e10e16b2-a594-48e7-b1cb-9a614cc7cedb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:31.106407 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1312/2651] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:35:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:35:31.117820 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b4d77c-a568-4dd0-bd4f-93ea505c07a9 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:31.118341 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b4d77c-a568-4dd0-bd4f-93ea505c07a9 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:31.118917 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b4d77c-a568-4dd0-bd4f-93ea505c07a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:31.155648 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-f2b4d77c-a568-4dd0-bd4f-93ea505c07a9 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:35:31.155648 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-f2b4d77c-a568-4dd0-bd4f-93ea505c07a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:31.170721 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1340/2652] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:35:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:35:31.174701 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10e16b2-a594-48e7-b1cb-9a614cc7cedb None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:31.185464 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10e16b2-a594-48e7-b1cb-9a614cc7cedb None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:31.190750 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f379e9-07c2-430b-ba81-6cc32f297c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:31.191446 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f379e9-07c2-430b-ba81-6cc32f297c27 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:31.191730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f379e9-07c2-430b-ba81-6cc32f297c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:31.192006 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f379e9-07c2-430b-ba81-6cc32f297c27 admin admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:31.192802 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f379e9-07c2-430b-ba81-6cc32f297c27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:35:31.193109 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f379e9-07c2-430b-ba81-6cc32f297c27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:31.195094 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f379e9-07c2-430b-ba81-6cc32f297c27 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:31.209950 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1313/2653] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:35:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:35:31.215137 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b4d77c-a568-4dd0-bd4f-93ea505c07a9 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:31.226662 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b4d77c-a568-4dd0-bd4f-93ea505c07a9 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:31.232158 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b84bb9-46ac-4b97-ae89-33dc8cc69746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:31.232901 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b84bb9-46ac-4b97-ae89-33dc8cc69746 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:31.233200 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b84bb9-46ac-4b97-ae89-33dc8cc69746 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:31.233471 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b84bb9-46ac-4b97-ae89-33dc8cc69746 admin admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:31.234206 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b84bb9-46ac-4b97-ae89-33dc8cc69746 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:31.236314 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b84bb9-46ac-4b97-ae89-33dc8cc69746 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:31.325993 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1341/2654] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:35:31 2022] POST /identity/v3/projects => generated 375 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:35:31.337727 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f379e9-07c2-430b-ba81-6cc32f297c27 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:31.343314 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:35:31.354976 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f379e9-07c2-430b-ba81-6cc32f297c27 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:31.464740 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97af8629-be76-4340-89c4-a0abf6e13121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:31.465450 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af8629-be76-4340-89c4-a0abf6e13121 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:31.465738 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af8629-be76-4340-89c4-a0abf6e13121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:31.466015 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af8629-be76-4340-89c4-a0abf6e13121 admin admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:31.466767 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97af8629-be76-4340-89c4-a0abf6e13121 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:31.468973 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97af8629-be76-4340-89c4-a0abf6e13121 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:31.621651 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1314/2655] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:35:31 2022] POST /identity/v3/users => generated 288 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:35:31.626229 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b84bb9-46ac-4b97-ae89-33dc8cc69746 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:31.638224 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b84bb9-46ac-4b97-ae89-33dc8cc69746 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:31.646710 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527f82b8-8885-444a-8379-39c32b620a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:31.648546 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f82b8-8885-444a-8379-39c32b620a48 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:31.649047 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f82b8-8885-444a-8379-39c32b620a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:31.649588 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f82b8-8885-444a-8379-39c32b620a48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:31.650998 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527f82b8-8885-444a-8379-39c32b620a48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:31.655330 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527f82b8-8885-444a-8379-39c32b620a48 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:31.683455 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1342/2656] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:35:31 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:35:31.685108 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97af8629-be76-4340-89c4-a0abf6e13121 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:31.706105 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97af8629-be76-4340-89c4-a0abf6e13121 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:31.715503 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18485b16-98d3-4445-a8b4-076b8f527470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:31.716716 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18485b16-98d3-4445-a8b4-076b8f527470 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:31.717297 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18485b16-98d3-4445-a8b4-076b8f527470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:31.717589 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18485b16-98d3-4445-a8b4-076b8f527470 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:31.718350 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18485b16-98d3-4445-a8b4-076b8f527470 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:31.720007 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18485b16-98d3-4445-a8b4-076b8f527470 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:31.732638 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1315/2657] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:35:31 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:35:31.738720 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527f82b8-8885-444a-8379-39c32b620a48 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:31.751175 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527f82b8-8885-444a-8379-39c32b620a48 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:31.759927 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010e5aae-a335-426e-b7e5-3345a66df48c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:31.761586 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e5aae-a335-426e-b7e5-3345a66df48c admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:31.762093 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e5aae-a335-426e-b7e5-3345a66df48c admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:31.762604 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e5aae-a335-426e-b7e5-3345a66df48c admin admin] PATH_INFO: `/v3/projects/0810160b01684fc08a30d9a5cdcb3520/users/a1a8b831f5854d4fbe863340b071a9a0/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:31.766866 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:35:31.791854 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010e5aae-a335-426e-b7e5-3345a66df48c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0810160b01684fc08a30d9a5cdcb3520, user_id=a1a8b831f5854d4fbe863340b071a9a0, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:31.797436 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010e5aae-a335-426e-b7e5-3345a66df48c admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:31.907323 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1343/2658] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:35:31 2022] PUT /identity/v3/projects/0810160b01684fc08a30d9a5cdcb3520/users/a1a8b831f5854d4fbe863340b071a9a0/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:35:31.919120 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab359bcb-5a13-4a95-ad00-a30e566d3f33 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:31.919601 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab359bcb-5a13-4a95-ad00-a30e566d3f33 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:31.920022 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab359bcb-5a13-4a95-ad00-a30e566d3f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:31.951677 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-ab359bcb-5a13-4a95-ad00-a30e566d3f33 None None] MFA Rules not processed for user `a1a8b831f5854d4fbe863340b071a9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:35:31.953078 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-ab359bcb-5a13-4a95-ad00-a30e566d3f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:32.041324 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1316/2659] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:35:31 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:35:32.046655 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010e5aae-a335-426e-b7e5-3345a66df48c admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:32.127016 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010e5aae-a335-426e-b7e5-3345a66df48c admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:32.133046 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d88fe0-326c-402d-b0d7-ac4b08a23a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:32.133731 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d88fe0-326c-402d-b0d7-ac4b08a23a30 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:32.134011 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d88fe0-326c-402d-b0d7-ac4b08a23a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:32.134283 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d88fe0-326c-402d-b0d7-ac4b08a23a30 admin admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:32.134987 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d88fe0-326c-402d-b0d7-ac4b08a23a30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:32.136982 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d88fe0-326c-402d-b0d7-ac4b08a23a30 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:32.222690 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1344/2660] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:35:32 2022] POST /identity/v3/projects => generated 373 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:35:32.227412 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab359bcb-5a13-4a95-ad00-a30e566d3f33 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:32.240307 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab359bcb-5a13-4a95-ad00-a30e566d3f33 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:32.307439 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31f7c2e-97c1-4e99-9d99-4289d73532dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:32.308280 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f7c2e-97c1-4e99-9d99-4289d73532dd admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:32.308727 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f7c2e-97c1-4e99-9d99-4289d73532dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:32.309107 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f7c2e-97c1-4e99-9d99-4289d73532dd admin admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:32.309931 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31f7c2e-97c1-4e99-9d99-4289d73532dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:32.312005 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31f7c2e-97c1-4e99-9d99-4289d73532dd admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:32.456561 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:35:32.464104 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1317/2661] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:35:32 2022] POST /identity/v3/users => generated 293 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:35:32.469209 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d88fe0-326c-402d-b0d7-ac4b08a23a30 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:32.488034 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d88fe0-326c-402d-b0d7-ac4b08a23a30 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:32.497185 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ac9f8-07ed-43e1-b132-8265a24b8fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:32.498024 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ac9f8-07ed-43e1-b132-8265a24b8fee admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:32.498396 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ac9f8-07ed-43e1-b132-8265a24b8fee admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:32.498848 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ac9f8-07ed-43e1-b132-8265a24b8fee admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:32.499791 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202ac9f8-07ed-43e1-b132-8265a24b8fee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:32.503465 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202ac9f8-07ed-43e1-b132-8265a24b8fee admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:32.526367 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1345/2662] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:35:32 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:35:32.531917 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31f7c2e-97c1-4e99-9d99-4289d73532dd admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:32.548655 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31f7c2e-97c1-4e99-9d99-4289d73532dd admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:32.555823 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff558d1-9db1-4203-9f8f-c3886f177f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:32.557161 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff558d1-9db1-4203-9f8f-c3886f177f2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:32.557638 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff558d1-9db1-4203-9f8f-c3886f177f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:32.558246 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff558d1-9db1-4203-9f8f-c3886f177f2b admin admin] PATH_INFO: `/v3/projects/3d268016e50e412296a4a89bf54540b2/users/d6b8b4c0a4504f7c89d36d4072364313/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:32.582781 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff558d1-9db1-4203-9f8f-c3886f177f2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d268016e50e412296a4a89bf54540b2, user_id=d6b8b4c0a4504f7c89d36d4072364313, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:32.588916 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff558d1-9db1-4203-9f8f-c3886f177f2b admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:32.732206 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1318/2663] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:35:32 2022] PUT /identity/v3/projects/3d268016e50e412296a4a89bf54540b2/users/d6b8b4c0a4504f7c89d36d4072364313/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:35:32.739175 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ac9f8-07ed-43e1-b132-8265a24b8fee admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:32.750112 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ac9f8-07ed-43e1-b132-8265a24b8fee admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:32.823633 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac0a18a-78a1-4a1c-9904-d6bd81918cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:32.824322 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0a18a-78a1-4a1c-9904-d6bd81918cfd admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:32.824633 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0a18a-78a1-4a1c-9904-d6bd81918cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:32.824923 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0a18a-78a1-4a1c-9904-d6bd81918cfd admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:32.825674 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac0a18a-78a1-4a1c-9904-d6bd81918cfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:32.827333 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac0a18a-78a1-4a1c-9904-d6bd81918cfd admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:32.840367 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1346/2664] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:35:32 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:35:32.845118 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff558d1-9db1-4203-9f8f-c3886f177f2b admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:32.858359 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff558d1-9db1-4203-9f8f-c3886f177f2b admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:32.865901 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3594bb-1f59-4d17-97a9-5b4d06fcaade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:32.867533 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3594bb-1f59-4d17-97a9-5b4d06fcaade admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:32.868040 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3594bb-1f59-4d17-97a9-5b4d06fcaade admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:32.868643 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3594bb-1f59-4d17-97a9-5b4d06fcaade admin admin] PATH_INFO: `/v3/projects/3d268016e50e412296a4a89bf54540b2/users/d6b8b4c0a4504f7c89d36d4072364313/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:32.872431 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3594bb-1f59-4d17-97a9-5b4d06fcaade admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d268016e50e412296a4a89bf54540b2, user_id=d6b8b4c0a4504f7c89d36d4072364313, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:32.877773 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3594bb-1f59-4d17-97a9-5b4d06fcaade admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:32.894745 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1319/2665] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:35:32 2022] PUT /identity/v3/projects/3d268016e50e412296a4a89bf54540b2/users/d6b8b4c0a4504f7c89d36d4072364313/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:35:32.907912 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bda2f0-857f-4d7e-bc5d-75e34ab071e3 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:32.908197 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bda2f0-857f-4d7e-bc5d-75e34ab071e3 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:32.908513 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bda2f0-857f-4d7e-bc5d-75e34ab071e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:32.940268 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-a7bda2f0-857f-4d7e-bc5d-75e34ab071e3 None None] MFA Rules not processed for user `d6b8b4c0a4504f7c89d36d4072364313`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:35:32.941582 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-a7bda2f0-857f-4d7e-bc5d-75e34ab071e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:32.993258 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:35:33.026369 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1347/2666] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:35:32 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:35:33.039724 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3594bb-1f59-4d17-97a9-5b4d06fcaade admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:33.041952 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-7d3594bb-1f59-4d17-97a9-5b4d06fcaade admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:33.133519 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3594bb-1f59-4d17-97a9-5b4d06fcaade admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:33.140573 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0530a35-d4a4-4e74-9c78-893c45688bae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:33.141937 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0530a35-d4a4-4e74-9c78-893c45688bae service cinder] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:33.142075 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0530a35-d4a4-4e74-9c78-893c45688bae service cinder] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:33.142512 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0530a35-d4a4-4e74-9c78-893c45688bae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:33.234630 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0530a35-d4a4-4e74-9c78-893c45688bae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:33.238873 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0530a35-d4a4-4e74-9c78-893c45688bae service cinder] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:33.253232 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1320/2667] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:35:33 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:35:39.757394 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bda2f0-857f-4d7e-bc5d-75e34ab071e3 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:39.850040 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bda2f0-857f-4d7e-bc5d-75e34ab071e3 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:39.857257 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dec5f9b-cb1d-4da3-a095-09b1c0297e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:39.858807 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec5f9b-cb1d-4da3-a095-09b1c0297e17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:39.859250 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec5f9b-cb1d-4da3-a095-09b1c0297e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:39.859690 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec5f9b-cb1d-4da3-a095-09b1c0297e17 admin admin] PATH_INFO: `/v3/users/a1a8b831f5854d4fbe863340b071a9a0` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:39.861632 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dec5f9b-cb1d-4da3-a095-09b1c0297e17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1a8b831f5854d4fbe863340b071a9a0)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:39.864806 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dec5f9b-cb1d-4da3-a095-09b1c0297e17 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:40.278225 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-4dec5f9b-cb1d-4da3-a095-09b1c0297e17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1a8b831f5854d4fbe863340b071a9a0'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:40.296697 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-4dec5f9b-cb1d-4da3-a095-09b1c0297e17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1a8b831f5854d4fbe863340b071a9a0'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:40.419967 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-4dec5f9b-cb1d-4da3-a095-09b1c0297e17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1a8b831f5854d4fbe863340b071a9a0'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:40.421030 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-4dec5f9b-cb1d-4da3-a095-09b1c0297e17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1a8b831f5854d4fbe863340b071a9a0'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:40.433039 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-4dec5f9b-cb1d-4da3-a095-09b1c0297e17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1a8b831f5854d4fbe863340b071a9a0'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:40.446052 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1348/2668] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:35:39 2022] DELETE /identity/v3/users/a1a8b831f5854d4fbe863340b071a9a0 => generated 0 bytes in 684 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:35:40.447920 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0421219-949e-4a48-b83d-3eb23b54e635 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:40.448818 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0421219-949e-4a48-b83d-3eb23b54e635 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:40.448985 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0421219-949e-4a48-b83d-3eb23b54e635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:40.471257 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-e0421219-949e-4a48-b83d-3eb23b54e635 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:35:40.472860 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-e0421219-949e-4a48-b83d-3eb23b54e635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:40.555819 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1321/2669] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:35:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:35:40.572240 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dec5f9b-cb1d-4da3-a095-09b1c0297e17 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:40.574541 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-4dec5f9b-cb1d-4da3-a095-09b1c0297e17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:40.679753 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dec5f9b-cb1d-4da3-a095-09b1c0297e17 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:40.686218 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b07922-783f-4507-95fa-6986f4a39759 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:40.687370 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b07922-783f-4507-95fa-6986f4a39759 service neutron] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:40.687880 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b07922-783f-4507-95fa-6986f4a39759 service neutron] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:40.688382 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b07922-783f-4507-95fa-6986f4a39759 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:40.706946 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b07922-783f-4507-95fa-6986f4a39759 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:40.710237 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b07922-783f-4507-95fa-6986f4a39759 service neutron] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:40.720022 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:35:40.727844 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1349/2670] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:35:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:35:41.336207 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0421219-949e-4a48-b83d-3eb23b54e635 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:41.337690 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-e0421219-949e-4a48-b83d-3eb23b54e635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:41.351248 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0421219-949e-4a48-b83d-3eb23b54e635 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:41.356916 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f07ba0b-9549-4abd-b9e8-b69a7d50371e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:41.358191 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f07ba0b-9549-4abd-b9e8-b69a7d50371e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:41.358479 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f07ba0b-9549-4abd-b9e8-b69a7d50371e admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:41.358751 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f07ba0b-9549-4abd-b9e8-b69a7d50371e admin admin] PATH_INFO: `/v3/projects/0810160b01684fc08a30d9a5cdcb3520` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:41.360412 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f07ba0b-9549-4abd-b9e8-b69a7d50371e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0810160b01684fc08a30d9a5cdcb3520)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:41.362583 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f07ba0b-9549-4abd-b9e8-b69a7d50371e admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:41.376155 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:35:41.519250 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-9f07ba0b-9549-4abd-b9e8-b69a7d50371e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0810160b01684fc08a30d9a5cdcb3520', 'request_id': 'req-9f07ba0b-9549-4abd-b9e8-b69a7d50371e'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:41.631518 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-9f07ba0b-9549-4abd-b9e8-b69a7d50371e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0810160b01684fc08a30d9a5cdcb3520', 'request_id': 'req-9f07ba0b-9549-4abd-b9e8-b69a7d50371e'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:41.639265 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-9f07ba0b-9549-4abd-b9e8-b69a7d50371e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0810160b01684fc08a30d9a5cdcb3520', 'request_id': 'req-9f07ba0b-9549-4abd-b9e8-b69a7d50371e'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:41.655329 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-9f07ba0b-9549-4abd-b9e8-b69a7d50371e admin admin] The token cache is being invalidate because project 0810160b01684fc08a30d9a5cdcb3520 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:35:41.655673 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-9f07ba0b-9549-4abd-b9e8-b69a7d50371e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:41.656358 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-9f07ba0b-9549-4abd-b9e8-b69a7d50371e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:41.658056 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1322/2671] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:35:41 2022] DELETE /identity/v3/projects/0810160b01684fc08a30d9a5cdcb3520 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:35:41.663438 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b07922-783f-4507-95fa-6986f4a39759 service neutron] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:41.675575 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b07922-783f-4507-95fa-6986f4a39759 service neutron] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:41.782418 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7748a170-9eda-43df-a0d2-dd84bdf1d5b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:41.783660 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7748a170-9eda-43df-a0d2-dd84bdf1d5b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:41.784161 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7748a170-9eda-43df-a0d2-dd84bdf1d5b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:41.784671 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7748a170-9eda-43df-a0d2-dd84bdf1d5b5 admin admin] PATH_INFO: `/v3/users/d6b8b4c0a4504f7c89d36d4072364313` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:41.787606 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7748a170-9eda-43df-a0d2-dd84bdf1d5b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6b8b4c0a4504f7c89d36d4072364313)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:41.792619 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7748a170-9eda-43df-a0d2-dd84bdf1d5b5 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:42.156373 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-7748a170-9eda-43df-a0d2-dd84bdf1d5b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6b8b4c0a4504f7c89d36d4072364313'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:42.194855 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-7748a170-9eda-43df-a0d2-dd84bdf1d5b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6b8b4c0a4504f7c89d36d4072364313'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:42.328788 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-7748a170-9eda-43df-a0d2-dd84bdf1d5b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6b8b4c0a4504f7c89d36d4072364313'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:42.329630 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-7748a170-9eda-43df-a0d2-dd84bdf1d5b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6b8b4c0a4504f7c89d36d4072364313'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:42.341320 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-7748a170-9eda-43df-a0d2-dd84bdf1d5b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6b8b4c0a4504f7c89d36d4072364313'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:42.350127 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1350/2672] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:35:41 2022] DELETE /identity/v3/users/d6b8b4c0a4504f7c89d36d4072364313 => generated 0 bytes in 687 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:35:43.008177 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f07ba0b-9549-4abd-b9e8-b69a7d50371e admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:43.011731 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-9f07ba0b-9549-4abd-b9e8-b69a7d50371e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:43.031457 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f07ba0b-9549-4abd-b9e8-b69a7d50371e admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:43.128841 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64bc496-d016-4339-a40e-43f7e5eafda4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:43.130197 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64bc496-d016-4339-a40e-43f7e5eafda4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:43.130527 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64bc496-d016-4339-a40e-43f7e5eafda4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:43.130875 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64bc496-d016-4339-a40e-43f7e5eafda4 admin admin] PATH_INFO: `/v3/projects/3d268016e50e412296a4a89bf54540b2` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:43.132358 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64bc496-d016-4339-a40e-43f7e5eafda4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d268016e50e412296a4a89bf54540b2)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:43.134726 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64bc496-d016-4339-a40e-43f7e5eafda4 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:43.289524 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-a64bc496-d016-4339-a40e-43f7e5eafda4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d268016e50e412296a4a89bf54540b2', 'request_id': 'req-a64bc496-d016-4339-a40e-43f7e5eafda4'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:43.385700 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-a64bc496-d016-4339-a40e-43f7e5eafda4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d268016e50e412296a4a89bf54540b2', 'request_id': 'req-a64bc496-d016-4339-a40e-43f7e5eafda4'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:43.394675 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-a64bc496-d016-4339-a40e-43f7e5eafda4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d268016e50e412296a4a89bf54540b2', 'request_id': 'req-a64bc496-d016-4339-a40e-43f7e5eafda4'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:43.409069 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-a64bc496-d016-4339-a40e-43f7e5eafda4 admin admin] The token cache is being invalidate because project 3d268016e50e412296a4a89bf54540b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:35:43.409503 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-a64bc496-d016-4339-a40e-43f7e5eafda4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:43.410534 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-a64bc496-d016-4339-a40e-43f7e5eafda4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:43.413338 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1323/2673] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:35:43 2022] DELETE /identity/v3/projects/3d268016e50e412296a4a89bf54540b2 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:35:43.419026 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e02234-b1f2-45fa-98a2-c59e6db7eb49 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:43.419961 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e02234-b1f2-45fa-98a2-c59e6db7eb49 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:43.419961 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e02234-b1f2-45fa-98a2-c59e6db7eb49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:43.441827 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-d9e02234-b1f2-45fa-98a2-c59e6db7eb49 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:35:43.443348 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-d9e02234-b1f2-45fa-98a2-c59e6db7eb49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:43.517517 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1351/2674] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:35:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:35:43.527771 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff026c4-7a6d-4a13-876d-bb08a13543ad None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:43.528160 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff026c4-7a6d-4a13-876d-bb08a13543ad None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:43.528551 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff026c4-7a6d-4a13-876d-bb08a13543ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:43.552674 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-2ff026c4-7a6d-4a13-876d-bb08a13543ad None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:35:43.554163 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-2ff026c4-7a6d-4a13-876d-bb08a13543ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:43.563700 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1324/2675] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:35:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:35:43.569311 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e02234-b1f2-45fa-98a2-c59e6db7eb49 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:43.583480 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e02234-b1f2-45fa-98a2-c59e6db7eb49 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:43.589262 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c281487-6406-4a50-9924-3e190d79e3e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:43.590015 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c281487-6406-4a50-9924-3e190d79e3e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:43.590342 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c281487-6406-4a50-9924-3e190d79e3e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:43.590715 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c281487-6406-4a50-9924-3e190d79e3e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:43.591559 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c281487-6406-4a50-9924-3e190d79e3e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:35:43.591874 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c281487-6406-4a50-9924-3e190d79e3e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:43.593665 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c281487-6406-4a50-9924-3e190d79e3e5 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:43.608552 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1352/2676] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:35:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:35:43.614749 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff026c4-7a6d-4a13-876d-bb08a13543ad None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:43.625517 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff026c4-7a6d-4a13-876d-bb08a13543ad None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:43.631942 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5d866c-a271-469d-9304-5b1bd0061150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:43.632844 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d866c-a271-469d-9304-5b1bd0061150 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:43.633299 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d866c-a271-469d-9304-5b1bd0061150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:43.633684 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d866c-a271-469d-9304-5b1bd0061150 admin admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:43.634538 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5d866c-a271-469d-9304-5b1bd0061150 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:43.636999 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5d866c-a271-469d-9304-5b1bd0061150 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:43.754742 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1325/2677] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:35:43 2022] POST /identity/v3/projects => generated 383 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:35:43.761085 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c281487-6406-4a50-9924-3e190d79e3e5 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:43.774426 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c281487-6406-4a50-9924-3e190d79e3e5 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:43.778854 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:35:43.854951 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788d30c5-0862-460c-a316-5f232532a4e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:43.856012 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d30c5-0862-460c-a316-5f232532a4e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:43.856470 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d30c5-0862-460c-a316-5f232532a4e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:43.856897 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d30c5-0862-460c-a316-5f232532a4e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:43.858026 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788d30c5-0862-460c-a316-5f232532a4e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:43.861258 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788d30c5-0862-460c-a316-5f232532a4e2 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:43.958073 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1353/2678] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:35:43 2022] POST /identity/v3/users => generated 292 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:35:43.963311 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5d866c-a271-469d-9304-5b1bd0061150 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:43.969363 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:35:43.975129 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5d866c-a271-469d-9304-5b1bd0061150 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:43.980206 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0388942-98f4-4215-8094-cedc1c9a8f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:43.980996 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0388942-98f4-4215-8094-cedc1c9a8f1f admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:43.981286 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0388942-98f4-4215-8094-cedc1c9a8f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:43.981563 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0388942-98f4-4215-8094-cedc1c9a8f1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:43.982310 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0388942-98f4-4215-8094-cedc1c9a8f1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:43.983969 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0388942-98f4-4215-8094-cedc1c9a8f1f admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:43.996666 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1326/2679] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:35:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:35:44.001241 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788d30c5-0862-460c-a316-5f232532a4e2 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:44.017400 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788d30c5-0862-460c-a316-5f232532a4e2 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:44.026124 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1223f7b1-af4c-47e0-bf41-6dbd1d87259a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:44.027184 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1223f7b1-af4c-47e0-bf41-6dbd1d87259a admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:44.027680 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1223f7b1-af4c-47e0-bf41-6dbd1d87259a admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:44.028170 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1223f7b1-af4c-47e0-bf41-6dbd1d87259a admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:44.029627 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1223f7b1-af4c-47e0-bf41-6dbd1d87259a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:44.032857 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1223f7b1-af4c-47e0-bf41-6dbd1d87259a admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:44.053906 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1354/2680] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:35:44 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:35:44.058279 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0388942-98f4-4215-8094-cedc1c9a8f1f admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:44.071665 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0388942-98f4-4215-8094-cedc1c9a8f1f admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:44.076994 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700af92f-17ca-4fef-a995-ed9c090f8ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:44.078019 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700af92f-17ca-4fef-a995-ed9c090f8ee7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:44.078431 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700af92f-17ca-4fef-a995-ed9c090f8ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:44.078872 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700af92f-17ca-4fef-a995-ed9c090f8ee7 admin admin] PATH_INFO: `/v3/projects/2ac9b07003e04195a452d9f27c3c70d5/users/fb371cda9d1148fbbbf91c41c76b9412/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:44.099082 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700af92f-17ca-4fef-a995-ed9c090f8ee7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ac9b07003e04195a452d9f27c3c70d5, user_id=fb371cda9d1148fbbbf91c41c76b9412, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:44.102213 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700af92f-17ca-4fef-a995-ed9c090f8ee7 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:44.222517 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1327/2681] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:35:44 2022] PUT /identity/v3/projects/2ac9b07003e04195a452d9f27c3c70d5/users/fb371cda9d1148fbbbf91c41c76b9412/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:35:44.244762 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58d004-d651-4947-b0d9-565cda9903f8 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:44.245194 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58d004-d651-4947-b0d9-565cda9903f8 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:44.245602 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58d004-d651-4947-b0d9-565cda9903f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:44.292523 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-9d58d004-d651-4947-b0d9-565cda9903f8 None None] MFA Rules not processed for user `fb371cda9d1148fbbbf91c41c76b9412`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:35:44.293927 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-9d58d004-d651-4947-b0d9-565cda9903f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:44.420013 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1355/2682] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:35:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:35:44.425482 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700af92f-17ca-4fef-a995-ed9c090f8ee7 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:44.509199 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700af92f-17ca-4fef-a995-ed9c090f8ee7 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:44.514844 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe2d434-37e6-42b3-ba55-322b016cb04d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:44.515693 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2d434-37e6-42b3-ba55-322b016cb04d admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:44.515890 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2d434-37e6-42b3-ba55-322b016cb04d admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:44.516164 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2d434-37e6-42b3-ba55-322b016cb04d admin admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:44.516933 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe2d434-37e6-42b3-ba55-322b016cb04d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:44.518954 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe2d434-37e6-42b3-ba55-322b016cb04d admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:44.651391 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1328/2683] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:35:44 2022] POST /identity/v3/projects => generated 383 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:35:44.657537 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d58d004-d651-4947-b0d9-565cda9903f8 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:44.670649 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d58d004-d651-4947-b0d9-565cda9903f8 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:44.760998 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cf9e56-aa1b-4d4b-b57a-a4e66d054de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:44.762217 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cf9e56-aa1b-4d4b-b57a-a4e66d054de8 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:44.762755 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cf9e56-aa1b-4d4b-b57a-a4e66d054de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:44.763336 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cf9e56-aa1b-4d4b-b57a-a4e66d054de8 admin admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:44.764828 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cf9e56-aa1b-4d4b-b57a-a4e66d054de8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:44.768912 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cf9e56-aa1b-4d4b-b57a-a4e66d054de8 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:44.922664 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1356/2684] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:35:44 2022] POST /identity/v3/users => generated 292 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:35:44.928108 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe2d434-37e6-42b3-ba55-322b016cb04d admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:44.942786 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe2d434-37e6-42b3-ba55-322b016cb04d admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:44.949165 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba29403b-f953-46af-bd6d-f5bb0a499527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:44.950378 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29403b-f953-46af-bd6d-f5bb0a499527 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:44.950931 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29403b-f953-46af-bd6d-f5bb0a499527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:44.951410 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29403b-f953-46af-bd6d-f5bb0a499527 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:44.952652 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba29403b-f953-46af-bd6d-f5bb0a499527 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:44.954681 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba29403b-f953-46af-bd6d-f5bb0a499527 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:44.969200 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1329/2685] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:35:44 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:35:44.975117 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cf9e56-aa1b-4d4b-b57a-a4e66d054de8 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:44.982968 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:35:44.988475 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cf9e56-aa1b-4d4b-b57a-a4e66d054de8 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:44.995186 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a42b600-f724-41b9-aaa1-ea162b8da219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:44.995919 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a42b600-f724-41b9-aaa1-ea162b8da219 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:44.996452 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a42b600-f724-41b9-aaa1-ea162b8da219 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:44.996648 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a42b600-f724-41b9-aaa1-ea162b8da219 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:44.997340 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a42b600-f724-41b9-aaa1-ea162b8da219 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:44.999351 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a42b600-f724-41b9-aaa1-ea162b8da219 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:45.011332 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1357/2686] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:35:44 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:35:45.017950 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba29403b-f953-46af-bd6d-f5bb0a499527 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:45.036674 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba29403b-f953-46af-bd6d-f5bb0a499527 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:45.042054 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9315e961-6e83-49ff-8008-4718d887d0c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:45.043300 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9315e961-6e83-49ff-8008-4718d887d0c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:45.043590 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9315e961-6e83-49ff-8008-4718d887d0c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:45.043985 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9315e961-6e83-49ff-8008-4718d887d0c2 admin admin] PATH_INFO: `/v3/projects/a56fd8ad6bd14d0daf683f5719e98ab3/users/40ea8b497f0340329ae5362832287609/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:45.065069 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9315e961-6e83-49ff-8008-4718d887d0c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a56fd8ad6bd14d0daf683f5719e98ab3, user_id=40ea8b497f0340329ae5362832287609, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:45.070397 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9315e961-6e83-49ff-8008-4718d887d0c2 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:45.146635 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1330/2687] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:35:45 2022] PUT /identity/v3/projects/a56fd8ad6bd14d0daf683f5719e98ab3/users/40ea8b497f0340329ae5362832287609/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:35:45.155864 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a50d9ae-13df-42c9-9cc8-c1caf9983dbc None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:45.156359 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a50d9ae-13df-42c9-9cc8-c1caf9983dbc None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:45.156886 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a50d9ae-13df-42c9-9cc8-c1caf9983dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:45.197295 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-1a50d9ae-13df-42c9-9cc8-c1caf9983dbc None None] MFA Rules not processed for user `40ea8b497f0340329ae5362832287609`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:35:45.198896 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-1a50d9ae-13df-42c9-9cc8-c1caf9983dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:45.303601 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1358/2688] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:35:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:35:45.308669 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9315e961-6e83-49ff-8008-4718d887d0c2 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:45.414196 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9315e961-6e83-49ff-8008-4718d887d0c2 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:45.419420 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5a784b-7bcb-427c-a784-c3aa2d4dacc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:45.420116 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a784b-7bcb-427c-a784-c3aa2d4dacc3 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:45.420428 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a784b-7bcb-427c-a784-c3aa2d4dacc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:45.420714 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a784b-7bcb-427c-a784-c3aa2d4dacc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:45.421432 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5a784b-7bcb-427c-a784-c3aa2d4dacc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:45.423418 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5a784b-7bcb-427c-a784-c3aa2d4dacc3 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:45.606796 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1331/2689] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:35:45 2022] POST /identity/v3/projects => generated 385 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:35:45.610375 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a50d9ae-13df-42c9-9cc8-c1caf9983dbc None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:45.622646 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a50d9ae-13df-42c9-9cc8-c1caf9983dbc None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:45.697370 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f570b-6be8-47e1-9355-9d0f4240495d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:45.698083 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f570b-6be8-47e1-9355-9d0f4240495d admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:45.698391 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f570b-6be8-47e1-9355-9d0f4240495d admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:45.698672 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f570b-6be8-47e1-9355-9d0f4240495d admin admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:45.699406 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73f570b-6be8-47e1-9355-9d0f4240495d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:45.701607 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73f570b-6be8-47e1-9355-9d0f4240495d admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:45.867450 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1359/2690] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:35:45 2022] POST /identity/v3/users => generated 299 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:35:45.872716 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5a784b-7bcb-427c-a784-c3aa2d4dacc3 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:45.883408 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:35:45.890640 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5a784b-7bcb-427c-a784-c3aa2d4dacc3 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:45.895826 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b7412a-d454-46b4-9f07-38222f7dee79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:45.896583 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b7412a-d454-46b4-9f07-38222f7dee79 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:45.896877 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b7412a-d454-46b4-9f07-38222f7dee79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:45.897710 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b7412a-d454-46b4-9f07-38222f7dee79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:45.898465 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b7412a-d454-46b4-9f07-38222f7dee79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:45.900108 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b7412a-d454-46b4-9f07-38222f7dee79 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:45.912411 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1332/2691] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:35:45 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:35:45.919207 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f570b-6be8-47e1-9355-9d0f4240495d admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:45.931241 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f570b-6be8-47e1-9355-9d0f4240495d admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:45.936346 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c3c5e9-f689-4a4e-b024-cee56eda477a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:45.937345 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c3c5e9-f689-4a4e-b024-cee56eda477a admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:45.937751 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c3c5e9-f689-4a4e-b024-cee56eda477a admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:45.938109 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c3c5e9-f689-4a4e-b024-cee56eda477a admin admin] PATH_INFO: `/v3/projects/1b6712f0ab0f4ea29a334da309596c8c/users/3aae197a6a9047529a0ee5235bf47cd1/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:45.958681 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c3c5e9-f689-4a4e-b024-cee56eda477a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b6712f0ab0f4ea29a334da309596c8c, user_id=3aae197a6a9047529a0ee5235bf47cd1, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:45.961651 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c3c5e9-f689-4a4e-b024-cee56eda477a admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:46.032639 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1360/2692] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:35:45 2022] PUT /identity/v3/projects/1b6712f0ab0f4ea29a334da309596c8c/users/3aae197a6a9047529a0ee5235bf47cd1/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:35:46.037730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b7412a-d454-46b4-9f07-38222f7dee79 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:46.048347 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b7412a-d454-46b4-9f07-38222f7dee79 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:46.114466 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214bb6af-1b22-48f1-8490-2351031f7d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:46.115274 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214bb6af-1b22-48f1-8490-2351031f7d4f admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:46.115483 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214bb6af-1b22-48f1-8490-2351031f7d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:46.115700 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214bb6af-1b22-48f1-8490-2351031f7d4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:46.116485 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214bb6af-1b22-48f1-8490-2351031f7d4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:46.118192 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214bb6af-1b22-48f1-8490-2351031f7d4f admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:46.131220 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1333/2693] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:35:46 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:35:46.135244 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c3c5e9-f689-4a4e-b024-cee56eda477a admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:46.147612 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c3c5e9-f689-4a4e-b024-cee56eda477a admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:46.153014 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441456f9-dae4-4061-b77f-c12af4ebfb57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:46.154059 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441456f9-dae4-4061-b77f-c12af4ebfb57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:46.154346 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441456f9-dae4-4061-b77f-c12af4ebfb57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:46.154627 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441456f9-dae4-4061-b77f-c12af4ebfb57 admin admin] PATH_INFO: `/v3/projects/1b6712f0ab0f4ea29a334da309596c8c/users/3aae197a6a9047529a0ee5235bf47cd1/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:46.156862 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441456f9-dae4-4061-b77f-c12af4ebfb57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b6712f0ab0f4ea29a334da309596c8c, user_id=3aae197a6a9047529a0ee5235bf47cd1, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:46.159579 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441456f9-dae4-4061-b77f-c12af4ebfb57 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:46.172250 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1361/2694] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:35:46 2022] PUT /identity/v3/projects/1b6712f0ab0f4ea29a334da309596c8c/users/3aae197a6a9047529a0ee5235bf47cd1/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:35:46.184454 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a804304-68fb-493a-95e0-33fc9aedec4d None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:46.184767 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a804304-68fb-493a-95e0-33fc9aedec4d None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:46.185025 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a804304-68fb-493a-95e0-33fc9aedec4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:46.216265 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-0a804304-68fb-493a-95e0-33fc9aedec4d None None] MFA Rules not processed for user `3aae197a6a9047529a0ee5235bf47cd1`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:35:46.217840 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-0a804304-68fb-493a-95e0-33fc9aedec4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:46.307063 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1334/2695] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:35:46 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:35:46.321079 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441456f9-dae4-4061-b77f-c12af4ebfb57 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:46.323190 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-441456f9-dae4-4061-b77f-c12af4ebfb57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:46.377910 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:35:46.412291 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441456f9-dae4-4061-b77f-c12af4ebfb57 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:46.418614 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6366f6-655d-423d-910b-d6ca46607e3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:46.428547 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6366f6-655d-423d-910b-d6ca46607e3b service cinder] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:46.428547 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6366f6-655d-423d-910b-d6ca46607e3b service cinder] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:46.428547 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6366f6-655d-423d-910b-d6ca46607e3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:46.495086 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6366f6-655d-423d-910b-d6ca46607e3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:46.496590 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6366f6-655d-423d-910b-d6ca46607e3b service cinder] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:46.508331 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1362/2696] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:35:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:35:46.879387 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a804304-68fb-493a-95e0-33fc9aedec4d None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:46.881217 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-0a804304-68fb-493a-95e0-33fc9aedec4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:46.970407 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a804304-68fb-493a-95e0-33fc9aedec4d None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:46.977283 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22df4911-9689-4bc3-97ee-e0ac20b8d63b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:46.978381 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df4911-9689-4bc3-97ee-e0ac20b8d63b service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:46.978861 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df4911-9689-4bc3-97ee-e0ac20b8d63b service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:46.979326 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df4911-9689-4bc3-97ee-e0ac20b8d63b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:46.982634 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-22df4911-9689-4bc3-97ee-e0ac20b8d63b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:47.042054 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=100 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:35:47.090944 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22df4911-9689-4bc3-97ee-e0ac20b8d63b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:47.090944 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22df4911-9689-4bc3-97ee-e0ac20b8d63b service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:47.101634 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1335/2697] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:35:46 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:35:48.619386 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6366f6-655d-423d-910b-d6ca46607e3b service cinder] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:48.626778 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-bb6366f6-655d-423d-910b-d6ca46607e3b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:48.647518 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6366f6-655d-423d-910b-d6ca46607e3b service cinder] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:48.647518 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f53f08e-c189-416c-a8e0-71ccb0286967 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:48.647518 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53f08e-c189-416c-a8e0-71ccb0286967 service cinder] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:48.647518 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53f08e-c189-416c-a8e0-71ccb0286967 service cinder] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:48.647989 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53f08e-c189-416c-a8e0-71ccb0286967 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:48.714890 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f53f08e-c189-416c-a8e0-71ccb0286967 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:48.716654 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f53f08e-c189-416c-a8e0-71ccb0286967 service cinder] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:48.729899 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1363/2698] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:35:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:35:50.521930 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22df4911-9689-4bc3-97ee-e0ac20b8d63b service neutron] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:50.533345 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22df4911-9689-4bc3-97ee-e0ac20b8d63b service neutron] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:50.537933 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fe26a2-0b31-4feb-8aa8-d6fae7fae2c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:50.538327 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe26a2-0b31-4feb-8aa8-d6fae7fae2c3 service cinder] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:50.538616 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe26a2-0b31-4feb-8aa8-d6fae7fae2c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:50.538961 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe26a2-0b31-4feb-8aa8-d6fae7fae2c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:50.549835 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fe26a2-0b31-4feb-8aa8-d6fae7fae2c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:50.551530 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fe26a2-0b31-4feb-8aa8-d6fae7fae2c3 service cinder] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:50.563471 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1336/2699] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:35:50 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:35:55.497964 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f53f08e-c189-416c-a8e0-71ccb0286967 service cinder] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:55.510711 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f53f08e-c189-416c-a8e0-71ccb0286967 service cinder] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:55.519378 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a1c22f-6f47-456d-bd59-af5604665320 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:55.519378 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1c22f-6f47-456d-bd59-af5604665320 service neutron] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:55.519378 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1c22f-6f47-456d-bd59-af5604665320 service neutron] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:55.519378 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1c22f-6f47-456d-bd59-af5604665320 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:55.522221 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-a3a1c22f-6f47-456d-bd59-af5604665320 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:55.609946 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a1c22f-6f47-456d-bd59-af5604665320 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:55.612804 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a1c22f-6f47-456d-bd59-af5604665320 service neutron] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:55.634538 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1364/2700] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:35:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:35:57.700540 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fe26a2-0b31-4feb-8aa8-d6fae7fae2c3 service cinder] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:57.712502 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fe26a2-0b31-4feb-8aa8-d6fae7fae2c3 service cinder] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:57.717740 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a4807-0b0e-4095-9977-b5ac90d33de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:57.718647 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a4807-0b0e-4095-9977-b5ac90d33de7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:57.718936 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a4807-0b0e-4095-9977-b5ac90d33de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:57.720441 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a4807-0b0e-4095-9977-b5ac90d33de7 admin admin] PATH_INFO: `/v3/users/fb371cda9d1148fbbbf91c41c76b9412` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:57.720575 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115a4807-0b0e-4095-9977-b5ac90d33de7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb371cda9d1148fbbbf91c41c76b9412)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:57.722677 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115a4807-0b0e-4095-9977-b5ac90d33de7 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:57.958907 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-115a4807-0b0e-4095-9977-b5ac90d33de7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb371cda9d1148fbbbf91c41c76b9412'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:57.982786 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-115a4807-0b0e-4095-9977-b5ac90d33de7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb371cda9d1148fbbbf91c41c76b9412'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:57.983732 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-115a4807-0b0e-4095-9977-b5ac90d33de7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb371cda9d1148fbbbf91c41c76b9412'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:58.001716 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-115a4807-0b0e-4095-9977-b5ac90d33de7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb371cda9d1148fbbbf91c41c76b9412'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:58.002330 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-115a4807-0b0e-4095-9977-b5ac90d33de7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb371cda9d1148fbbbf91c41c76b9412'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:58.102291 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1337/2701] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:35:57 2022] DELETE /identity/v3/users/fb371cda9d1148fbbbf91c41c76b9412 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:35:58.107282 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3ffb9-ae73-47b1-a355-f32894e3ae72 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:58.107832 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3ffb9-ae73-47b1-a355-f32894e3ae72 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:58.108161 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3ffb9-ae73-47b1-a355-f32894e3ae72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:58.127087 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-f5c3ffb9-ae73-47b1-a355-f32894e3ae72 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:35:58.128490 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-f5c3ffb9-ae73-47b1-a355-f32894e3ae72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:58.154337 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=100 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:35:58.222166 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1365/2702] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:35:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:35:58.234536 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a4807-0b0e-4095-9977-b5ac90d33de7 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:58.239879 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-115a4807-0b0e-4095-9977-b5ac90d33de7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:58.329015 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a4807-0b0e-4095-9977-b5ac90d33de7 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:58.334240 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3b03cc-83aa-434f-9782-557f301762a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:58.335141 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3b03cc-83aa-434f-9782-557f301762a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:58.335388 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3b03cc-83aa-434f-9782-557f301762a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:58.335737 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3b03cc-83aa-434f-9782-557f301762a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:58.347683 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3b03cc-83aa-434f-9782-557f301762a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:58.351511 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3b03cc-83aa-434f-9782-557f301762a9 service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:58.366802 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1338/2703] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:35:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:35:58.932979 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c3ffb9-ae73-47b1-a355-f32894e3ae72 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:58.934490 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-f5c3ffb9-ae73-47b1-a355-f32894e3ae72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:35:58.967587 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c3ffb9-ae73-47b1-a355-f32894e3ae72 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:58.975321 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ab5d45-e521-459b-ae7c-5f92fb6a2349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:58.976227 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab5d45-e521-459b-ae7c-5f92fb6a2349 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:58.980198 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab5d45-e521-459b-ae7c-5f92fb6a2349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:58.980772 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab5d45-e521-459b-ae7c-5f92fb6a2349 admin admin] PATH_INFO: `/v3/projects/2ac9b07003e04195a452d9f27c3c70d5` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:58.982347 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ab5d45-e521-459b-ae7c-5f92fb6a2349 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ac9b07003e04195a452d9f27c3c70d5)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:58.984655 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ab5d45-e521-459b-ae7c-5f92fb6a2349 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:59.177723 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-75ab5d45-e521-459b-ae7c-5f92fb6a2349 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ac9b07003e04195a452d9f27c3c70d5', 'request_id': 'req-75ab5d45-e521-459b-ae7c-5f92fb6a2349'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:59.188233 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-75ab5d45-e521-459b-ae7c-5f92fb6a2349 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ac9b07003e04195a452d9f27c3c70d5', 'request_id': 'req-75ab5d45-e521-459b-ae7c-5f92fb6a2349'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:59.196095 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-75ab5d45-e521-459b-ae7c-5f92fb6a2349 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ac9b07003e04195a452d9f27c3c70d5', 'request_id': 'req-75ab5d45-e521-459b-ae7c-5f92fb6a2349'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:59.311757 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-75ab5d45-e521-459b-ae7c-5f92fb6a2349 admin admin] The token cache is being invalidate because project 2ac9b07003e04195a452d9f27c3c70d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:35:59.312204 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-75ab5d45-e521-459b-ae7c-5f92fb6a2349 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:59.313285 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-75ab5d45-e521-459b-ae7c-5f92fb6a2349 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:59.319375 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1366/2704] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:35:58 2022] DELETE /identity/v3/projects/2ac9b07003e04195a452d9f27c3c70d5 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:35:59.320145 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3b03cc-83aa-434f-9782-557f301762a9 service neutron] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:35:59.340782 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3b03cc-83aa-434f-9782-557f301762a9 service neutron] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:35:59.377723 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:35:59.437377 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46350123-d990-4ee2-b984-a77a596ffcfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:35:59.443549 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46350123-d990-4ee2-b984-a77a596ffcfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:35:59.443549 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46350123-d990-4ee2-b984-a77a596ffcfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:35:59.443549 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46350123-d990-4ee2-b984-a77a596ffcfa admin admin] PATH_INFO: `/v3/users/40ea8b497f0340329ae5362832287609` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:35:59.443549 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46350123-d990-4ee2-b984-a77a596ffcfa admin admin] RBAC: Authorizing `identity:delete_user(user_id=40ea8b497f0340329ae5362832287609)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:35:59.443549 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46350123-d990-4ee2-b984-a77a596ffcfa admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:35:59.692164 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-46350123-d990-4ee2-b984-a77a596ffcfa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40ea8b497f0340329ae5362832287609'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:59.710703 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-46350123-d990-4ee2-b984-a77a596ffcfa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40ea8b497f0340329ae5362832287609'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:59.711571 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-46350123-d990-4ee2-b984-a77a596ffcfa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40ea8b497f0340329ae5362832287609'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:59.720302 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-46350123-d990-4ee2-b984-a77a596ffcfa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40ea8b497f0340329ae5362832287609'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:59.721358 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-46350123-d990-4ee2-b984-a77a596ffcfa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40ea8b497f0340329ae5362832287609'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:35:59.837531 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1339/2705] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:35:59 2022] DELETE /identity/v3/users/40ea8b497f0340329ae5362832287609 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:36:00.517924 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ab5d45-e521-459b-ae7c-5f92fb6a2349 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:36:00.520847 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-75ab5d45-e521-459b-ae7c-5f92fb6a2349 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:36:00.555910 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ab5d45-e521-459b-ae7c-5f92fb6a2349 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:36:00.634933 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4786823b-4ec2-4197-9ac7-6338107ffafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:36:00.636011 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4786823b-4ec2-4197-9ac7-6338107ffafd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:36:00.636433 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4786823b-4ec2-4197-9ac7-6338107ffafd admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:36:00.636765 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4786823b-4ec2-4197-9ac7-6338107ffafd admin admin] PATH_INFO: `/v3/projects/a56fd8ad6bd14d0daf683f5719e98ab3` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:36:00.638153 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4786823b-4ec2-4197-9ac7-6338107ffafd admin admin] RBAC: Authorizing `identity:delete_project(project_id=a56fd8ad6bd14d0daf683f5719e98ab3)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:36:00.640296 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4786823b-4ec2-4197-9ac7-6338107ffafd admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:36:00.809927 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:36:00.812199 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-4786823b-4ec2-4197-9ac7-6338107ffafd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a56fd8ad6bd14d0daf683f5719e98ab3', 'request_id': 'req-4786823b-4ec2-4197-9ac7-6338107ffafd'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:36:00.825301 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-4786823b-4ec2-4197-9ac7-6338107ffafd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a56fd8ad6bd14d0daf683f5719e98ab3', 'request_id': 'req-4786823b-4ec2-4197-9ac7-6338107ffafd'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:36:00.835942 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-4786823b-4ec2-4197-9ac7-6338107ffafd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a56fd8ad6bd14d0daf683f5719e98ab3', 'request_id': 'req-4786823b-4ec2-4197-9ac7-6338107ffafd'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:36:00.930089 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-4786823b-4ec2-4197-9ac7-6338107ffafd admin admin] The token cache is being invalidate because project a56fd8ad6bd14d0daf683f5719e98ab3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:36:00.930601 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-4786823b-4ec2-4197-9ac7-6338107ffafd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:36:00.931763 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-4786823b-4ec2-4197-9ac7-6338107ffafd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:36:00.934676 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1367/2706] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:36:00 2022] DELETE /identity/v3/projects/a56fd8ad6bd14d0daf683f5719e98ab3 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:36:00.940734 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46350123-d990-4ee2-b984-a77a596ffcfa admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:36:00.944823 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-46350123-d990-4ee2-b984-a77a596ffcfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:36:00.969011 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46350123-d990-4ee2-b984-a77a596ffcfa admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:36:01.065977 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f223da1-db1a-405f-8d41-057a72327500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:36:01.067414 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f223da1-db1a-405f-8d41-057a72327500 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:36:01.067699 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f223da1-db1a-405f-8d41-057a72327500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:36:01.067980 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f223da1-db1a-405f-8d41-057a72327500 admin admin] PATH_INFO: `/v3/users/3aae197a6a9047529a0ee5235bf47cd1` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:36:01.069373 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f223da1-db1a-405f-8d41-057a72327500 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3aae197a6a9047529a0ee5235bf47cd1)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:36:01.071372 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f223da1-db1a-405f-8d41-057a72327500 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:36:01.288008 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-7f223da1-db1a-405f-8d41-057a72327500 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3aae197a6a9047529a0ee5235bf47cd1'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:36:01.291528 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:36:01.310133 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-7f223da1-db1a-405f-8d41-057a72327500 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3aae197a6a9047529a0ee5235bf47cd1'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:36:01.311156 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-7f223da1-db1a-405f-8d41-057a72327500 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3aae197a6a9047529a0ee5235bf47cd1'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:36:01.327381 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-7f223da1-db1a-405f-8d41-057a72327500 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3aae197a6a9047529a0ee5235bf47cd1'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:36:01.328142 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-7f223da1-db1a-405f-8d41-057a72327500 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3aae197a6a9047529a0ee5235bf47cd1'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:36:01.415653 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1340/2707] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:36:00 2022] DELETE /identity/v3/users/3aae197a6a9047529a0ee5235bf47cd1 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:36:02.080601 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4786823b-4ec2-4197-9ac7-6338107ffafd admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:36:02.082167 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-4786823b-4ec2-4197-9ac7-6338107ffafd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:36:02.117384 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4786823b-4ec2-4197-9ac7-6338107ffafd admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:36:02.245613 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757ab21b-5dea-43e2-8a66-aca9cd81914f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:36:02.249400 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ab21b-5dea-43e2-8a66-aca9cd81914f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:36:02.250201 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ab21b-5dea-43e2-8a66-aca9cd81914f admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:36:02.251157 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ab21b-5dea-43e2-8a66-aca9cd81914f admin admin] PATH_INFO: `/v3/projects/1b6712f0ab0f4ea29a334da309596c8c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:36:02.253787 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757ab21b-5dea-43e2-8a66-aca9cd81914f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b6712f0ab0f4ea29a334da309596c8c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:36:02.258864 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757ab21b-5dea-43e2-8a66-aca9cd81914f admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:36:02.420912 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-757ab21b-5dea-43e2-8a66-aca9cd81914f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b6712f0ab0f4ea29a334da309596c8c', 'request_id': 'req-757ab21b-5dea-43e2-8a66-aca9cd81914f'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:36:02.428689 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-757ab21b-5dea-43e2-8a66-aca9cd81914f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b6712f0ab0f4ea29a334da309596c8c', 'request_id': 'req-757ab21b-5dea-43e2-8a66-aca9cd81914f'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:36:02.435435 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-757ab21b-5dea-43e2-8a66-aca9cd81914f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b6712f0ab0f4ea29a334da309596c8c', 'request_id': 'req-757ab21b-5dea-43e2-8a66-aca9cd81914f'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:36:02.569913 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-757ab21b-5dea-43e2-8a66-aca9cd81914f admin admin] The token cache is being invalidate because project 1b6712f0ab0f4ea29a334da309596c8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:36:02.570321 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-757ab21b-5dea-43e2-8a66-aca9cd81914f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:36:02.571040 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-757ab21b-5dea-43e2-8a66-aca9cd81914f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:36:02.575890 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1368/2708] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:36:02 2022] DELETE /identity/v3/projects/1b6712f0ab0f4ea29a334da309596c8c => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:36:02.580236 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9896a49-29e1-42d8-8802-a6886fcbd4fd None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:36:02.580905 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9896a49-29e1-42d8-8802-a6886fcbd4fd None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:36:02.583229 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9896a49-29e1-42d8-8802-a6886fcbd4fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:36:02.627087 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-a9896a49-29e1-42d8-8802-a6886fcbd4fd None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:36:02.630423 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-a9896a49-29e1-42d8-8802-a6886fcbd4fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:36:02.631269 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd009d-52df-407d-9fc9-8a056aa48329 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:36:02.631563 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd009d-52df-407d-9fc9-8a056aa48329 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:36:02.631838 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd009d-52df-407d-9fc9-8a056aa48329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:36:02.663031 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-a8dd009d-52df-407d-9fc9-8a056aa48329 None None] MFA Rules not processed for user `78a853ef1e51411e88ec7984693be865`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:36:02.664332 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-a8dd009d-52df-407d-9fc9-8a056aa48329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:36:02.751658 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1369/2709] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:36:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:36:02.773352 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1341/2710] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:36:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:36:02.784825 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c726ae19-a9ca-4eea-b1bc-27293a9862b6 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:36:02.785122 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c726ae19-a9ca-4eea-b1bc-27293a9862b6 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:36:02.786069 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c726ae19-a9ca-4eea-b1bc-27293a9862b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:36:02.819647 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-c726ae19-a9ca-4eea-b1bc-27293a9862b6 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:36:02.821049 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-c726ae19-a9ca-4eea-b1bc-27293a9862b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:36:02.833145 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1370/2711] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:36:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:36:02.836849 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9896a49-29e1-42d8-8802-a6886fcbd4fd None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:36:02.851638 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9896a49-29e1-42d8-8802-a6886fcbd4fd None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:36:02.862630 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4cf4fd-d43c-433c-8d24-cb428b8b09c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:36:02.863467 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4cf4fd-d43c-433c-8d24-cb428b8b09c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:36:02.863973 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4cf4fd-d43c-433c-8d24-cb428b8b09c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:36:02.864475 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4cf4fd-d43c-433c-8d24-cb428b8b09c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:36:02.865435 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4cf4fd-d43c-433c-8d24-cb428b8b09c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:36:02.865820 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4cf4fd-d43c-433c-8d24-cb428b8b09c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:36:02.867668 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4cf4fd-d43c-433c-8d24-cb428b8b09c1 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:36:02.888455 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1342/2712] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:36:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:36:02.888739 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c726ae19-a9ca-4eea-b1bc-27293a9862b6 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:36:02.893030 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:36:02.897983 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c726ae19-a9ca-4eea-b1bc-27293a9862b6 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:36:02.903339 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625cc218-8009-4afe-84f1-4afd348b4618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:36:02.904252 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625cc218-8009-4afe-84f1-4afd348b4618 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:36:02.904637 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625cc218-8009-4afe-84f1-4afd348b4618 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:36:02.905012 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625cc218-8009-4afe-84f1-4afd348b4618 admin admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:36:02.905983 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625cc218-8009-4afe-84f1-4afd348b4618 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:36:02.908073 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625cc218-8009-4afe-84f1-4afd348b4618 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:36:03.031781 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1371/2713] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:36:02 2022] POST /identity/v3/projects => generated 379 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:36:03.034030 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4cf4fd-d43c-433c-8d24-cb428b8b09c1 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:36:03.063476 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4cf4fd-d43c-433c-8d24-cb428b8b09c1 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:36:03.210328 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de1b4e7-0365-4f2c-b7fd-c8ef564b94f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:36:03.211226 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de1b4e7-0365-4f2c-b7fd-c8ef564b94f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:36:03.211616 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de1b4e7-0365-4f2c-b7fd-c8ef564b94f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:36:03.211999 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de1b4e7-0365-4f2c-b7fd-c8ef564b94f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:36:03.212861 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de1b4e7-0365-4f2c-b7fd-c8ef564b94f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:36:03.215634 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de1b4e7-0365-4f2c-b7fd-c8ef564b94f0 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:36:03.342420 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1343/2714] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:36:03 2022] POST /identity/v3/users => generated 290 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:36:03.356300 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625cc218-8009-4afe-84f1-4afd348b4618 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:36:03.400992 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625cc218-8009-4afe-84f1-4afd348b4618 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:36:03.408188 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e160302e-b7eb-4e8c-b7cd-1e93ba8613b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:36:03.408933 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e160302e-b7eb-4e8c-b7cd-1e93ba8613b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:36:03.409222 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e160302e-b7eb-4e8c-b7cd-1e93ba8613b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:36:03.409502 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e160302e-b7eb-4e8c-b7cd-1e93ba8613b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:36:03.410258 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e160302e-b7eb-4e8c-b7cd-1e93ba8613b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:36:03.412873 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e160302e-b7eb-4e8c-b7cd-1e93ba8613b7 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:36:03.433760 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1372/2715] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:36:03 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:36:03.438560 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de1b4e7-0365-4f2c-b7fd-c8ef564b94f0 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:36:03.451756 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de1b4e7-0365-4f2c-b7fd-c8ef564b94f0 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:36:03.457075 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56535902-7a57-4c20-9288-4a31a9b8809e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:36:03.458404 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56535902-7a57-4c20-9288-4a31a9b8809e admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:36:03.467828 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56535902-7a57-4c20-9288-4a31a9b8809e admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:36:03.467828 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56535902-7a57-4c20-9288-4a31a9b8809e admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:36:03.467828 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56535902-7a57-4c20-9288-4a31a9b8809e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:36:03.467828 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56535902-7a57-4c20-9288-4a31a9b8809e admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:36:03.475386 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1344/2716] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:36:03 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:36:03.481560 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e160302e-b7eb-4e8c-b7cd-1e93ba8613b7 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:36:03.499653 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e160302e-b7eb-4e8c-b7cd-1e93ba8613b7 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:36:03.512439 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43770a77-447d-4aa7-986a-c2ea350eb6a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:36:03.514096 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43770a77-447d-4aa7-986a-c2ea350eb6a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:36:03.515525 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43770a77-447d-4aa7-986a-c2ea350eb6a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:36:03.515525 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43770a77-447d-4aa7-986a-c2ea350eb6a6 admin admin] PATH_INFO: `/v3/projects/cfbaadcdab8348e79dd89d42efe5fe1c/users/8d522e55ac82498b8714c75dee8ceac0/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:36:03.541519 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43770a77-447d-4aa7-986a-c2ea350eb6a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfbaadcdab8348e79dd89d42efe5fe1c, user_id=8d522e55ac82498b8714c75dee8ceac0, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:36:03.545003 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43770a77-447d-4aa7-986a-c2ea350eb6a6 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:36:03.641573 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1373/2717] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:36:03 2022] PUT /identity/v3/projects/cfbaadcdab8348e79dd89d42efe5fe1c/users/8d522e55ac82498b8714c75dee8ceac0/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:36:03.729723 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf8de0f-e9b7-40ed-b261-84c85ccaaf07 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:36:03.730302 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf8de0f-e9b7-40ed-b261-84c85ccaaf07 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:36:03.730476 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf8de0f-e9b7-40ed-b261-84c85ccaaf07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:36:03.771698 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-5bf8de0f-e9b7-40ed-b261-84c85ccaaf07 None None] MFA Rules not processed for user `8d522e55ac82498b8714c75dee8ceac0`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:36:03.774487 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-5bf8de0f-e9b7-40ed-b261-84c85ccaaf07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:36:03.801778 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:36:03.896442 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1345/2718] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:36:03 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:36:03.915390 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43770a77-447d-4aa7-986a-c2ea350eb6a6 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:36:03.920347 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-43770a77-447d-4aa7-986a-c2ea350eb6a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:36:04.060152 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43770a77-447d-4aa7-986a-c2ea350eb6a6 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:36:04.065198 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043ac389-e3e1-47b7-b17b-94b77a23f021 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:36:04.066266 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043ac389-e3e1-47b7-b17b-94b77a23f021 service cinder] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:36:04.067337 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043ac389-e3e1-47b7-b17b-94b77a23f021 service cinder] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:36:04.068071 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043ac389-e3e1-47b7-b17b-94b77a23f021 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:36:04.087113 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043ac389-e3e1-47b7-b17b-94b77a23f021 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:36:04.089122 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043ac389-e3e1-47b7-b17b-94b77a23f021 service cinder] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:36:04.107757 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1374/2719] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:36:03 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:36:05.060183 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf8de0f-e9b7-40ed-b261-84c85ccaaf07 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:36:05.063238 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-5bf8de0f-e9b7-40ed-b261-84c85ccaaf07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:36:05.179874 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf8de0f-e9b7-40ed-b261-84c85ccaaf07 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:36:05.185331 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dcafe5-ff22-44d0-88f8-67a1b6ef07e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:36:05.186507 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dcafe5-ff22-44d0-88f8-67a1b6ef07e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:36:05.186792 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dcafe5-ff22-44d0-88f8-67a1b6ef07e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:36:05.187074 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dcafe5-ff22-44d0-88f8-67a1b6ef07e2 admin admin] PATH_INFO: `/v3/users/113b17a680c640f19b0f402f371c6cbb` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:36:05.188923 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dcafe5-ff22-44d0-88f8-67a1b6ef07e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=113b17a680c640f19b0f402f371c6cbb)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:36:05.190926 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dcafe5-ff22-44d0-88f8-67a1b6ef07e2 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:36:05.716261 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-42dcafe5-ff22-44d0-88f8-67a1b6ef07e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '113b17a680c640f19b0f402f371c6cbb'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:36:05.731949 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-42dcafe5-ff22-44d0-88f8-67a1b6ef07e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '113b17a680c640f19b0f402f371c6cbb'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:36:05.732724 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-42dcafe5-ff22-44d0-88f8-67a1b6ef07e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '113b17a680c640f19b0f402f371c6cbb'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:36:05.741137 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-42dcafe5-ff22-44d0-88f8-67a1b6ef07e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '113b17a680c640f19b0f402f371c6cbb'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:36:05.741886 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-42dcafe5-ff22-44d0-88f8-67a1b6ef07e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '113b17a680c640f19b0f402f371c6cbb'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:36:05.913341 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1346/2720] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:36:05 2022] DELETE /identity/v3/users/113b17a680c640f19b0f402f371c6cbb => generated 0 bytes in 854 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:36:06.261904 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043ac389-e3e1-47b7-b17b-94b77a23f021 service cinder] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:36:06.276214 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043ac389-e3e1-47b7-b17b-94b77a23f021 service cinder] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:36:06.359020 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb9400d-643c-44f4-be58-350d93773024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:36:06.359915 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb9400d-643c-44f4-be58-350d93773024 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:36:06.360197 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb9400d-643c-44f4-be58-350d93773024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:36:06.360504 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb9400d-643c-44f4-be58-350d93773024 admin admin] PATH_INFO: `/v3/projects/65efebfec3f0401190b3116e52d634e8` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:36:06.361716 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb9400d-643c-44f4-be58-350d93773024 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65efebfec3f0401190b3116e52d634e8)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:36:06.365625 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb9400d-643c-44f4-be58-350d93773024 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:36:06.576948 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-aeb9400d-643c-44f4-be58-350d93773024 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65efebfec3f0401190b3116e52d634e8', 'request_id': 'req-aeb9400d-643c-44f4-be58-350d93773024'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:36:06.579150 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:36:06.584835 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-aeb9400d-643c-44f4-be58-350d93773024 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65efebfec3f0401190b3116e52d634e8', 'request_id': 'req-aeb9400d-643c-44f4-be58-350d93773024'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:36:06.591136 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-aeb9400d-643c-44f4-be58-350d93773024 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65efebfec3f0401190b3116e52d634e8', 'request_id': 'req-aeb9400d-643c-44f4-be58-350d93773024'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:36:06.734695 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-aeb9400d-643c-44f4-be58-350d93773024 admin admin] The token cache is being invalidate because project 65efebfec3f0401190b3116e52d634e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:36:06.735014 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-aeb9400d-643c-44f4-be58-350d93773024 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:36:06.735681 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-aeb9400d-643c-44f4-be58-350d93773024 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:36:06.737317 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1375/2721] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:36:06 2022] DELETE /identity/v3/projects/65efebfec3f0401190b3116e52d634e8 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:36:06.742501 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dcafe5-ff22-44d0-88f8-67a1b6ef07e2 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:36:06.744169 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-42dcafe5-ff22-44d0-88f8-67a1b6ef07e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:36:06.759092 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dcafe5-ff22-44d0-88f8-67a1b6ef07e2 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:36:06.809620 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:36:06.826850 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0153fb8-4255-4af8-b0e5-89154c28c317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:36:06.827823 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0153fb8-4255-4af8-b0e5-89154c28c317 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:36:06.828069 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0153fb8-4255-4af8-b0e5-89154c28c317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:36:06.828373 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0153fb8-4255-4af8-b0e5-89154c28c317 admin admin] PATH_INFO: `/v3/users/02a53b7a3b3741378f324fbcebde804a` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:36:06.829816 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0153fb8-4255-4af8-b0e5-89154c28c317 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02a53b7a3b3741378f324fbcebde804a)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:36:06.831814 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0153fb8-4255-4af8-b0e5-89154c28c317 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:36:07.057717 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-f0153fb8-4255-4af8-b0e5-89154c28c317 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02a53b7a3b3741378f324fbcebde804a'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:36:07.074537 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-f0153fb8-4255-4af8-b0e5-89154c28c317 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02a53b7a3b3741378f324fbcebde804a'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:36:07.075903 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-f0153fb8-4255-4af8-b0e5-89154c28c317 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02a53b7a3b3741378f324fbcebde804a'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:36:07.091602 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-f0153fb8-4255-4af8-b0e5-89154c28c317 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02a53b7a3b3741378f324fbcebde804a'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:36:07.091602 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-f0153fb8-4255-4af8-b0e5-89154c28c317 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02a53b7a3b3741378f324fbcebde804a'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:36:07.198092 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1347/2722] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:36:06 2022] DELETE /identity/v3/users/02a53b7a3b3741378f324fbcebde804a => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:36:07.680157 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb9400d-643c-44f4-be58-350d93773024 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:36:07.683764 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-aeb9400d-643c-44f4-be58-350d93773024 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:36:07.727671 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb9400d-643c-44f4-be58-350d93773024 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:36:07.765271 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0153fb8-4255-4af8-b0e5-89154c28c317 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:36:07.767666 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-f0153fb8-4255-4af8-b0e5-89154c28c317 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:36:07.821555 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134d3c0f-ebeb-4e62-a775-6b03e3fb7172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:36:07.822463 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d3c0f-ebeb-4e62-a775-6b03e3fb7172 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:36:07.822771 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d3c0f-ebeb-4e62-a775-6b03e3fb7172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:36:07.823104 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d3c0f-ebeb-4e62-a775-6b03e3fb7172 admin admin] PATH_INFO: `/v3/projects/1a18e7fcbdab4e1b86e201ac4a3f7aa3` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:36:07.824546 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134d3c0f-ebeb-4e62-a775-6b03e3fb7172 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a18e7fcbdab4e1b86e201ac4a3f7aa3)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:36:07.829890 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134d3c0f-ebeb-4e62-a775-6b03e3fb7172 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:36:07.865542 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0153fb8-4255-4af8-b0e5-89154c28c317 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:36:07.871033 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd4bfc6-7dd8-419b-9743-3ce5b88b1671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:36:07.871961 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd4bfc6-7dd8-419b-9743-3ce5b88b1671 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:36:07.873232 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd4bfc6-7dd8-419b-9743-3ce5b88b1671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:36:07.873232 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd4bfc6-7dd8-419b-9743-3ce5b88b1671 admin admin] PATH_INFO: `/v3/users/8d522e55ac82498b8714c75dee8ceac0` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:36:07.874195 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd4bfc6-7dd8-419b-9743-3ce5b88b1671 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d522e55ac82498b8714c75dee8ceac0)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:36:07.876226 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd4bfc6-7dd8-419b-9743-3ce5b88b1671 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:36:07.944188 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-134d3c0f-ebeb-4e62-a775-6b03e3fb7172 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a18e7fcbdab4e1b86e201ac4a3f7aa3', 'request_id': 'req-134d3c0f-ebeb-4e62-a775-6b03e3fb7172'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:36:07.951948 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-134d3c0f-ebeb-4e62-a775-6b03e3fb7172 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a18e7fcbdab4e1b86e201ac4a3f7aa3', 'request_id': 'req-134d3c0f-ebeb-4e62-a775-6b03e3fb7172'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:36:07.959435 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-134d3c0f-ebeb-4e62-a775-6b03e3fb7172 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a18e7fcbdab4e1b86e201ac4a3f7aa3', 'request_id': 'req-134d3c0f-ebeb-4e62-a775-6b03e3fb7172'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:36:08.042384 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-134d3c0f-ebeb-4e62-a775-6b03e3fb7172 admin admin] The token cache is being invalidate because project 1a18e7fcbdab4e1b86e201ac4a3f7aa3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:36:08.042743 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-134d3c0f-ebeb-4e62-a775-6b03e3fb7172 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:36:08.043523 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-134d3c0f-ebeb-4e62-a775-6b03e3fb7172 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:36:08.045568 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1376/2723] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:36:07 2022] DELETE /identity/v3/projects/1a18e7fcbdab4e1b86e201ac4a3f7aa3 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:36:08.126122 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-9bd4bfc6-7dd8-419b-9743-3ce5b88b1671 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d522e55ac82498b8714c75dee8ceac0'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:36:08.147798 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:36:08.150467 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-9bd4bfc6-7dd8-419b-9743-3ce5b88b1671 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d522e55ac82498b8714c75dee8ceac0'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:36:08.152012 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-9bd4bfc6-7dd8-419b-9743-3ce5b88b1671 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d522e55ac82498b8714c75dee8ceac0'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:36:08.165915 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-9bd4bfc6-7dd8-419b-9743-3ce5b88b1671 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d522e55ac82498b8714c75dee8ceac0'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:36:08.167170 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-9bd4bfc6-7dd8-419b-9743-3ce5b88b1671 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d522e55ac82498b8714c75dee8ceac0'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:36:08.266825 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1348/2724] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:36:07 2022] DELETE /identity/v3/users/8d522e55ac82498b8714c75dee8ceac0 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:36:08.276052 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60353a-7930-40ac-a592-1ffc99d58e81 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:36:08.276052 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60353a-7930-40ac-a592-1ffc99d58e81 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:36:08.276052 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60353a-7930-40ac-a592-1ffc99d58e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:36:08.307006 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-4a60353a-7930-40ac-a592-1ffc99d58e81 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:36:08.309141 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-4a60353a-7930-40ac-a592-1ffc99d58e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:36:08.443295 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1377/2725] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:36:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:36:08.450615 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd4bfc6-7dd8-419b-9743-3ce5b88b1671 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:36:08.452444 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-9bd4bfc6-7dd8-419b-9743-3ce5b88b1671 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:36:08.538692 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd4bfc6-7dd8-419b-9743-3ce5b88b1671 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:36:08.545223 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f141e8-f52c-4037-934e-4757b06d1a9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:36:08.546496 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f141e8-f52c-4037-934e-4757b06d1a9a service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:36:08.546978 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f141e8-f52c-4037-934e-4757b06d1a9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:36:08.547450 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f141e8-f52c-4037-934e-4757b06d1a9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:36:08.560932 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f141e8-f52c-4037-934e-4757b06d1a9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:36:08.563623 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f141e8-f52c-4037-934e-4757b06d1a9a service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:36:08.574342 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1349/2726] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:36:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:36:09.120367 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a60353a-7930-40ac-a592-1ffc99d58e81 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:36:09.122360 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-4a60353a-7930-40ac-a592-1ffc99d58e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:36:09.145199 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a60353a-7930-40ac-a592-1ffc99d58e81 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:36:09.153741 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8bfb81-3e74-43d7-b074-4ca89633ae97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:36:09.155404 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8bfb81-3e74-43d7-b074-4ca89633ae97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:36:09.155896 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8bfb81-3e74-43d7-b074-4ca89633ae97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:36:09.156379 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8bfb81-3e74-43d7-b074-4ca89633ae97 admin admin] PATH_INFO: `/v3/projects/cfbaadcdab8348e79dd89d42efe5fe1c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:36:09.159006 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8bfb81-3e74-43d7-b074-4ca89633ae97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfbaadcdab8348e79dd89d42efe5fe1c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:36:09.163760 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8bfb81-3e74-43d7-b074-4ca89633ae97 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:36:09.175044 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:36:09.334081 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-ed8bfb81-3e74-43d7-b074-4ca89633ae97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfbaadcdab8348e79dd89d42efe5fe1c', 'request_id': 'req-ed8bfb81-3e74-43d7-b074-4ca89633ae97'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:36:09.345801 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-ed8bfb81-3e74-43d7-b074-4ca89633ae97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfbaadcdab8348e79dd89d42efe5fe1c', 'request_id': 'req-ed8bfb81-3e74-43d7-b074-4ca89633ae97'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:36:09.353757 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-ed8bfb81-3e74-43d7-b074-4ca89633ae97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfbaadcdab8348e79dd89d42efe5fe1c', 'request_id': 'req-ed8bfb81-3e74-43d7-b074-4ca89633ae97'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:36:09.427486 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-ed8bfb81-3e74-43d7-b074-4ca89633ae97 admin admin] The token cache is being invalidate because project cfbaadcdab8348e79dd89d42efe5fe1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:36:09.427795 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-ed8bfb81-3e74-43d7-b074-4ca89633ae97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:36:09.428582 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-ed8bfb81-3e74-43d7-b074-4ca89633ae97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:36:09.434792 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1378/2727] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:36:09 2022] DELETE /identity/v3/projects/cfbaadcdab8348e79dd89d42efe5fe1c => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:36:09.436017 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f6732c-ddb2-42c3-8964-a3fd38979d38 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:36:09.436770 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f6732c-ddb2-42c3-8964-a3fd38979d38 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:36:09.436770 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f6732c-ddb2-42c3-8964-a3fd38979d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:36:09.457652 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-54f6732c-ddb2-42c3-8964-a3fd38979d38 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:36:09.459300 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-54f6732c-ddb2-42c3-8964-a3fd38979d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:36:09.533790 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1350/2728] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:36:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:36:09.543609 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9d9b49-8acc-414a-b9ff-07757a091add None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:36:09.544100 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9d9b49-8acc-414a-b9ff-07757a091add None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:36:09.544670 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9d9b49-8acc-414a-b9ff-07757a091add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:36:09.579676 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-fa9d9b49-8acc-414a-b9ff-07757a091add None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:36:09.581929 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-fa9d9b49-8acc-414a-b9ff-07757a091add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:36:09.592733 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1379/2729] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:36:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:36:09.597270 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f6732c-ddb2-42c3-8964-a3fd38979d38 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:36:09.608971 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f6732c-ddb2-42c3-8964-a3fd38979d38 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:36:09.614900 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bd1e89-ca6b-4c1d-aa5b-2eee7d154abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:36:09.615619 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd1e89-ca6b-4c1d-aa5b-2eee7d154abd admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:36:09.615905 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd1e89-ca6b-4c1d-aa5b-2eee7d154abd admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:36:09.616195 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd1e89-ca6b-4c1d-aa5b-2eee7d154abd admin admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:36:09.617041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bd1e89-ca6b-4c1d-aa5b-2eee7d154abd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:36:09.617354 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bd1e89-ca6b-4c1d-aa5b-2eee7d154abd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:36:09.619201 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bd1e89-ca6b-4c1d-aa5b-2eee7d154abd admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:36:09.637841 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1351/2730] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:36:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:36:09.639815 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9d9b49-8acc-414a-b9ff-07757a091add None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:36:09.657275 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9d9b49-8acc-414a-b9ff-07757a091add None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:36:09.664250 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1170cad0-3184-41e5-9284-c0764600a1e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:36:09.665086 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1170cad0-3184-41e5-9284-c0764600a1e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:36:09.665448 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1170cad0-3184-41e5-9284-c0764600a1e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:36:09.665788 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1170cad0-3184-41e5-9284-c0764600a1e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:36:09.666628 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1170cad0-3184-41e5-9284-c0764600a1e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:36:09.668867 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1170cad0-3184-41e5-9284-c0764600a1e4 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:36:09.821070 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1380/2731] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:36:09 2022] POST /identity/v3/projects => generated 373 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:36:09.827755 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bd1e89-ca6b-4c1d-aa5b-2eee7d154abd admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:36:09.839945 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bd1e89-ca6b-4c1d-aa5b-2eee7d154abd admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:36:09.930077 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbef338-39ed-4572-9b9d-a6a50b437874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:36:09.930792 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbef338-39ed-4572-9b9d-a6a50b437874 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:36:09.931055 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbef338-39ed-4572-9b9d-a6a50b437874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:36:09.931323 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbef338-39ed-4572-9b9d-a6a50b437874 admin admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:36:09.932035 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbef338-39ed-4572-9b9d-a6a50b437874 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:36:09.934109 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbef338-39ed-4572-9b9d-a6a50b437874 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:36:10.032453 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:36:10.040044 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1352/2732] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:36:09 2022] POST /identity/v3/users => generated 287 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:36:10.044828 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1170cad0-3184-41e5-9284-c0764600a1e4 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:36:10.061803 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1170cad0-3184-41e5-9284-c0764600a1e4 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:36:10.067672 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc795e8e-3ce6-4db0-962b-a480a7538123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:36:10.068408 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc795e8e-3ce6-4db0-962b-a480a7538123 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:36:10.068751 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc795e8e-3ce6-4db0-962b-a480a7538123 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:36:10.069032 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc795e8e-3ce6-4db0-962b-a480a7538123 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:36:10.069777 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc795e8e-3ce6-4db0-962b-a480a7538123 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:36:10.071391 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc795e8e-3ce6-4db0-962b-a480a7538123 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:36:10.085433 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1381/2733] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:36:10 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:36:10.089632 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbef338-39ed-4572-9b9d-a6a50b437874 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:36:10.099756 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbef338-39ed-4572-9b9d-a6a50b437874 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:36:10.104469 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3463db9d-cdf5-4e1c-9978-24727925129d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:36:10.105158 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3463db9d-cdf5-4e1c-9978-24727925129d admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:36:10.105432 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3463db9d-cdf5-4e1c-9978-24727925129d admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:36:10.105709 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3463db9d-cdf5-4e1c-9978-24727925129d admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:36:10.106445 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3463db9d-cdf5-4e1c-9978-24727925129d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:36:10.108040 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3463db9d-cdf5-4e1c-9978-24727925129d admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:36:10.119078 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1353/2734] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:36:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:36:10.124424 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc795e8e-3ce6-4db0-962b-a480a7538123 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:36:10.136334 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc795e8e-3ce6-4db0-962b-a480a7538123 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:36:10.141084 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192952e0-592a-4de1-8fa0-46aff20271f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:36:10.141987 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192952e0-592a-4de1-8fa0-46aff20271f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:36:10.142275 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192952e0-592a-4de1-8fa0-46aff20271f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:36:10.142553 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192952e0-592a-4de1-8fa0-46aff20271f7 admin admin] PATH_INFO: `/v3/projects/090fe72cdd1a486f9da51253e153c45a/users/cdac600e644741ce83b25e684d1ea6d8/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:36:10.167449 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192952e0-592a-4de1-8fa0-46aff20271f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=090fe72cdd1a486f9da51253e153c45a, user_id=cdac600e644741ce83b25e684d1ea6d8, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:36:10.170198 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192952e0-592a-4de1-8fa0-46aff20271f7 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:36:10.249538 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1382/2735] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:36:10 2022] PUT /identity/v3/projects/090fe72cdd1a486f9da51253e153c45a/users/cdac600e644741ce83b25e684d1ea6d8/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:36:10.263331 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d9cdd-1a4a-4565-8ecb-6e8cbad8e9e7 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:36:10.263888 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d9cdd-1a4a-4565-8ecb-6e8cbad8e9e7 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:36:10.264169 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d9cdd-1a4a-4565-8ecb-6e8cbad8e9e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:36:10.299494 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-521d9cdd-1a4a-4565-8ecb-6e8cbad8e9e7 None None] MFA Rules not processed for user `cdac600e644741ce83b25e684d1ea6d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:36:10.300777 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-521d9cdd-1a4a-4565-8ecb-6e8cbad8e9e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:36:10.383282 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1354/2736] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:36:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:36:10.391952 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192952e0-592a-4de1-8fa0-46aff20271f7 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:36:10.394288 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-192952e0-592a-4de1-8fa0-46aff20271f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:36:10.490567 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192952e0-592a-4de1-8fa0-46aff20271f7 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:36:10.494638 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71583103-e2e4-4f0e-82d2-35eda0457156 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:36:10.495316 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71583103-e2e4-4f0e-82d2-35eda0457156 service cinder] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:36:10.495603 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71583103-e2e4-4f0e-82d2-35eda0457156 service cinder] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:36:10.495884 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71583103-e2e4-4f0e-82d2-35eda0457156 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:36:10.506936 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71583103-e2e4-4f0e-82d2-35eda0457156 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:36:10.508690 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71583103-e2e4-4f0e-82d2-35eda0457156 service cinder] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:36:10.522822 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1383/2737] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:36:10 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:36:12.492198 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa3b322-e814-4f8a-a540-1014a8033565 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:36:12.492821 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa3b322-e814-4f8a-a540-1014a8033565 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:36:12.492821 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa3b322-e814-4f8a-a540-1014a8033565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:36:12.512108 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-6aa3b322-e814-4f8a-a540-1014a8033565 None None] MFA Rules not processed for user `cdac600e644741ce83b25e684d1ea6d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:36:12.513428 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-6aa3b322-e814-4f8a-a540-1014a8033565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:36:12.520742 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1355/2738] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:36:12 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:36:20.516293 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=75,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:36:22.505971 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=40 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:36:33.187470 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac53af6-eca8-48e7-a0c7-ae4b86aa1770 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:36:33.188036 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac53af6-eca8-48e7-a0c7-ae4b86aa1770 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:36:33.188036 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac53af6-eca8-48e7-a0c7-ae4b86aa1770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:36:33.208781 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-eac53af6-eca8-48e7-a0c7-ae4b86aa1770 None None] MFA Rules not processed for user `fcfcd9f134e74b09bfdac337c81c8a07`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:36:33.210164 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-eac53af6-eca8-48e7-a0c7-ae4b86aa1770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:36:33.285125 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1384/2739] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Jun 21 18:36:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:36:33.451420 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa3b322-e814-4f8a-a540-1014a8033565 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:36:33.452928 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-6aa3b322-e814-4f8a-a540-1014a8033565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:36:33.522966 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa3b322-e814-4f8a-a540-1014a8033565 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:36:33.528297 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc43d24-e017-40c1-a42a-4ed44ae121de service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e0d25565f1543d9a7577288fc09119f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:36:33.529057 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc43d24-e017-40c1-a42a-4ed44ae121de service swift] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:36:33.529409 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc43d24-e017-40c1-a42a-4ed44ae121de service swift] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:36:33.529755 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc43d24-e017-40c1-a42a-4ed44ae121de service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:36:33.544468 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc43d24-e017-40c1-a42a-4ed44ae121de service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:36:33.546327 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc43d24-e017-40c1-a42a-4ed44ae121de service swift] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:36:33.555745 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1356/2740] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Jun 21 18:36:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 18:36:34.386661 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4300b551-a6fa-49b5-963a-d5d3d78bba14 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:36:34.387198 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4300b551-a6fa-49b5-963a-d5d3d78bba14 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:36:34.387198 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4300b551-a6fa-49b5-963a-d5d3d78bba14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:36:34.406629 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-4300b551-a6fa-49b5-963a-d5d3d78bba14 None None] MFA Rules not processed for user `fcfcd9f134e74b09bfdac337c81c8a07`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:36:34.407861 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-4300b551-a6fa-49b5-963a-d5d3d78bba14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:36:34.416881 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1385/2741] 10.222.0.23 () {58 vars in 967 bytes} [Tue Jun 21 18:36:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:36:34.431761 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc43d24-e017-40c1-a42a-4ed44ae121de service swift] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:36:34.441994 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc43d24-e017-40c1-a42a-4ed44ae121de service swift] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:36:34.448545 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56211d63-cf8e-4d16-a32b-c90f1bf9f14f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e0d25565f1543d9a7577288fc09119f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:36:34.453785 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56211d63-cf8e-4d16-a32b-c90f1bf9f14f service swift] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:36:34.453785 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56211d63-cf8e-4d16-a32b-c90f1bf9f14f service swift] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:36:34.453785 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56211d63-cf8e-4d16-a32b-c90f1bf9f14f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:36:34.459663 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56211d63-cf8e-4d16-a32b-c90f1bf9f14f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:36:34.462015 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56211d63-cf8e-4d16-a32b-c90f1bf9f14f service swift] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:36:34.471532 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1357/2742] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Jun 21 18:36:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 18:36:35.151765 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce1728-3893-402b-b0d5-ddb89444ac5d None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:36:35.154940 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce1728-3893-402b-b0d5-ddb89444ac5d None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:36:35.154940 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce1728-3893-402b-b0d5-ddb89444ac5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:36:35.172383 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-fbce1728-3893-402b-b0d5-ddb89444ac5d None None] MFA Rules not processed for user `fcfcd9f134e74b09bfdac337c81c8a07`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:36:35.173662 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-fbce1728-3893-402b-b0d5-ddb89444ac5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:36:35.181749 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1386/2743] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Jun 21 18:36:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:36:35.195392 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56211d63-cf8e-4d16-a32b-c90f1bf9f14f service swift] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:36:35.206087 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56211d63-cf8e-4d16-a32b-c90f1bf9f14f service swift] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:36:35.212494 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5896b2-08da-41b8-8b3d-99e179734c87 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e0d25565f1543d9a7577288fc09119f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:36:35.213142 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5896b2-08da-41b8-8b3d-99e179734c87 service swift] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:36:35.213420 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5896b2-08da-41b8-8b3d-99e179734c87 service swift] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:36:35.213696 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5896b2-08da-41b8-8b3d-99e179734c87 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:36:35.222260 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5896b2-08da-41b8-8b3d-99e179734c87 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:36:35.225026 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5896b2-08da-41b8-8b3d-99e179734c87 service swift] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:36:35.233520 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1358/2744] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Jun 21 18:36:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 18:36:35.900275 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd48667b-0d1f-4197-af79-8adc9947ed4c None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:36:35.900797 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd48667b-0d1f-4197-af79-8adc9947ed4c None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:36:35.900899 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd48667b-0d1f-4197-af79-8adc9947ed4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:36:35.919469 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-bd48667b-0d1f-4197-af79-8adc9947ed4c None None] MFA Rules not processed for user `fcfcd9f134e74b09bfdac337c81c8a07`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:36:35.920700 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-bd48667b-0d1f-4197-af79-8adc9947ed4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:36:35.928901 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1387/2745] 10.222.0.23 () {58 vars in 967 bytes} [Tue Jun 21 18:36:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:36:35.941992 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5896b2-08da-41b8-8b3d-99e179734c87 service swift] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:36:35.952939 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5896b2-08da-41b8-8b3d-99e179734c87 service swift] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:36:35.958243 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d4503c-b551-4fc8-adb2-03237721a1dc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e0d25565f1543d9a7577288fc09119f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:36:35.958973 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d4503c-b551-4fc8-adb2-03237721a1dc service swift] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:36:35.959258 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d4503c-b551-4fc8-adb2-03237721a1dc service swift] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:36:35.959542 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d4503c-b551-4fc8-adb2-03237721a1dc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:36:35.968841 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d4503c-b551-4fc8-adb2-03237721a1dc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:36:35.970434 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d4503c-b551-4fc8-adb2-03237721a1dc service swift] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:36:35.979808 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1359/2746] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Jun 21 18:36:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 18:36:38.682801 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd48667b-0d1f-4197-af79-8adc9947ed4c None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:36:38.746394 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd48667b-0d1f-4197-af79-8adc9947ed4c None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:36:38.750927 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a76191-6792-4c1f-9d7e-a5d25c6b2240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:36:38.751813 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a76191-6792-4c1f-9d7e-a5d25c6b2240 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:36:38.752097 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a76191-6792-4c1f-9d7e-a5d25c6b2240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:36:38.752376 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a76191-6792-4c1f-9d7e-a5d25c6b2240 admin admin] PATH_INFO: `/v3/users/cdac600e644741ce83b25e684d1ea6d8` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:36:38.753627 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a76191-6792-4c1f-9d7e-a5d25c6b2240 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdac600e644741ce83b25e684d1ea6d8)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:36:38.755848 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a76191-6792-4c1f-9d7e-a5d25c6b2240 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:36:38.975961 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-90a76191-6792-4c1f-9d7e-a5d25c6b2240 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdac600e644741ce83b25e684d1ea6d8'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:36:38.987840 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-90a76191-6792-4c1f-9d7e-a5d25c6b2240 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdac600e644741ce83b25e684d1ea6d8'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:36:39.075549 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-90a76191-6792-4c1f-9d7e-a5d25c6b2240 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdac600e644741ce83b25e684d1ea6d8'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:36:39.076194 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-90a76191-6792-4c1f-9d7e-a5d25c6b2240 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdac600e644741ce83b25e684d1ea6d8'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:36:39.084099 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-90a76191-6792-4c1f-9d7e-a5d25c6b2240 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdac600e644741ce83b25e684d1ea6d8'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:36:39.090142 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1388/2747] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:36:38 2022] DELETE /identity/v3/users/cdac600e644741ce83b25e684d1ea6d8 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:36:39.094811 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66447600-3128-4e3e-9ae5-8536843c78c4 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:36:39.095326 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66447600-3128-4e3e-9ae5-8536843c78c4 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:36:39.095427 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66447600-3128-4e3e-9ae5-8536843c78c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:36:39.113084 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-66447600-3128-4e3e-9ae5-8536843c78c4 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:36:39.114347 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-66447600-3128-4e3e-9ae5-8536843c78c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:36:39.178496 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1360/2748] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:36:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:36:39.188307 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a76191-6792-4c1f-9d7e-a5d25c6b2240 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:36:39.189867 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-90a76191-6792-4c1f-9d7e-a5d25c6b2240 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:36:39.196361 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:36:39.268480 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a76191-6792-4c1f-9d7e-a5d25c6b2240 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:36:39.272301 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745e3230-503a-4c72-9def-d73e5b6b9472 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:36:39.272992 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745e3230-503a-4c72-9def-d73e5b6b9472 service neutron] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:36:39.273277 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745e3230-503a-4c72-9def-d73e5b6b9472 service neutron] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:36:39.273554 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745e3230-503a-4c72-9def-d73e5b6b9472 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:36:39.283974 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745e3230-503a-4c72-9def-d73e5b6b9472 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:36:39.285447 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745e3230-503a-4c72-9def-d73e5b6b9472 service neutron] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:36:39.296339 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1389/2749] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:36:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:36:39.678195 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66447600-3128-4e3e-9ae5-8536843c78c4 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:36:39.679861 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-66447600-3128-4e3e-9ae5-8536843c78c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:36:39.692823 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66447600-3128-4e3e-9ae5-8536843c78c4 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:36:39.698901 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5382886-2ff0-4662-9531-5faf46f26844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:36:39.700572 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5382886-2ff0-4662-9531-5faf46f26844 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:36:39.701089 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5382886-2ff0-4662-9531-5faf46f26844 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:36:39.701771 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5382886-2ff0-4662-9531-5faf46f26844 admin admin] PATH_INFO: `/v3/projects/090fe72cdd1a486f9da51253e153c45a` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:36:39.704228 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5382886-2ff0-4662-9531-5faf46f26844 admin admin] RBAC: Authorizing `identity:delete_project(project_id=090fe72cdd1a486f9da51253e153c45a)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:36:39.708170 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5382886-2ff0-4662-9531-5faf46f26844 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:36:39.846266 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:36:39.849577 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-e5382886-2ff0-4662-9531-5faf46f26844 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '090fe72cdd1a486f9da51253e153c45a', 'request_id': 'req-e5382886-2ff0-4662-9531-5faf46f26844'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:36:39.921617 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-e5382886-2ff0-4662-9531-5faf46f26844 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '090fe72cdd1a486f9da51253e153c45a', 'request_id': 'req-e5382886-2ff0-4662-9531-5faf46f26844'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:36:39.930117 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-e5382886-2ff0-4662-9531-5faf46f26844 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '090fe72cdd1a486f9da51253e153c45a', 'request_id': 'req-e5382886-2ff0-4662-9531-5faf46f26844'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:36:39.943108 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-e5382886-2ff0-4662-9531-5faf46f26844 admin admin] The token cache is being invalidate because project 090fe72cdd1a486f9da51253e153c45a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:36:39.943401 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-e5382886-2ff0-4662-9531-5faf46f26844 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:36:39.944088 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-e5382886-2ff0-4662-9531-5faf46f26844 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:36:39.945756 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1361/2750] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:36:39 2022] DELETE /identity/v3/projects/090fe72cdd1a486f9da51253e153c45a => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:36:49.435319 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=24 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:36:49.939758 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:DELETE=4,keystone:SELECT=7,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:37:24.260138 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745e3230-503a-4c72-9def-d73e5b6b9472 service neutron] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:37:24.261675 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-745e3230-503a-4c72-9def-d73e5b6b9472 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:37:24.276418 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745e3230-503a-4c72-9def-d73e5b6b9472 service neutron] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:37:24.340014 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae92fdc-1772-4421-ae68-33b64880b3ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:37:24.340756 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae92fdc-1772-4421-ae68-33b64880b3ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:37:24.341056 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae92fdc-1772-4421-ae68-33b64880b3ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:37:24.341337 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae92fdc-1772-4421-ae68-33b64880b3ff admin admin] PATH_INFO: `/v3/users/78a853ef1e51411e88ec7984693be865` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:37:24.342607 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae92fdc-1772-4421-ae68-33b64880b3ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=78a853ef1e51411e88ec7984693be865)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:37:24.344558 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae92fdc-1772-4421-ae68-33b64880b3ff admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:37:24.609110 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-fae92fdc-1772-4421-ae68-33b64880b3ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78a853ef1e51411e88ec7984693be865'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:37:24.622845 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-fae92fdc-1772-4421-ae68-33b64880b3ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78a853ef1e51411e88ec7984693be865'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:37:24.709622 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-fae92fdc-1772-4421-ae68-33b64880b3ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78a853ef1e51411e88ec7984693be865'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:37:24.710258 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-fae92fdc-1772-4421-ae68-33b64880b3ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78a853ef1e51411e88ec7984693be865'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:37:24.718349 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-fae92fdc-1772-4421-ae68-33b64880b3ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78a853ef1e51411e88ec7984693be865'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:37:24.724670 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1390/2751] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:37:24 2022] DELETE /identity/v3/users/78a853ef1e51411e88ec7984693be865 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:37:24.990400 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5382886-2ff0-4662-9531-5faf46f26844 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:37:24.992228 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-e5382886-2ff0-4662-9531-5faf46f26844 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:37:25.007812 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5382886-2ff0-4662-9531-5faf46f26844 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:37:25.073201 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f5ce8b-117e-435b-82c3-456ea7ee27fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:37:25.074119 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5ce8b-117e-435b-82c3-456ea7ee27fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:37:25.074395 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5ce8b-117e-435b-82c3-456ea7ee27fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:37:25.074676 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5ce8b-117e-435b-82c3-456ea7ee27fe admin admin] PATH_INFO: `/v3/projects/20016956d3e2409cb41b02f7f0617daa` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:37:25.076910 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f5ce8b-117e-435b-82c3-456ea7ee27fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=20016956d3e2409cb41b02f7f0617daa)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:37:25.078946 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f5ce8b-117e-435b-82c3-456ea7ee27fe admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:37:25.203018 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-98f5ce8b-117e-435b-82c3-456ea7ee27fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20016956d3e2409cb41b02f7f0617daa', 'request_id': 'req-98f5ce8b-117e-435b-82c3-456ea7ee27fe'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:37:25.293395 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-98f5ce8b-117e-435b-82c3-456ea7ee27fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20016956d3e2409cb41b02f7f0617daa', 'request_id': 'req-98f5ce8b-117e-435b-82c3-456ea7ee27fe'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:37:25.300111 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-98f5ce8b-117e-435b-82c3-456ea7ee27fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20016956d3e2409cb41b02f7f0617daa', 'request_id': 'req-98f5ce8b-117e-435b-82c3-456ea7ee27fe'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:37:25.312875 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-98f5ce8b-117e-435b-82c3-456ea7ee27fe admin admin] The token cache is being invalidate because project 20016956d3e2409cb41b02f7f0617daa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:37:25.313527 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-98f5ce8b-117e-435b-82c3-456ea7ee27fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:37:25.314010 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-98f5ce8b-117e-435b-82c3-456ea7ee27fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:37:25.316577 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1362/2752] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:37:24 2022] DELETE /identity/v3/projects/20016956d3e2409cb41b02f7f0617daa => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:37:30.039966 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae92fdc-1772-4421-ae68-33b64880b3ff admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:37:30.041470 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-fae92fdc-1772-4421-ae68-33b64880b3ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:37:30.101001 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae92fdc-1772-4421-ae68-33b64880b3ff admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:37:30.133797 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:37:30.151999 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fc4969-fbdb-4121-b9f2-1f0a001f2bcf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:37:30.152767 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fc4969-fbdb-4121-b9f2-1f0a001f2bcf service cinder] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:37:30.153061 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fc4969-fbdb-4121-b9f2-1f0a001f2bcf service cinder] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:37:30.153337 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fc4969-fbdb-4121-b9f2-1f0a001f2bcf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:37:30.155246 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-c7fc4969-fbdb-4121-b9f2-1f0a001f2bcf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:37:30.234717 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fc4969-fbdb-4121-b9f2-1f0a001f2bcf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:37:30.236564 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fc4969-fbdb-4121-b9f2-1f0a001f2bcf service cinder] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:37:30.249701 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1391/2753] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:37:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:37:33.193843 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f5ce8b-117e-435b-82c3-456ea7ee27fe admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:37:33.195372 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-98f5ce8b-117e-435b-82c3-456ea7ee27fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:37:33.254309 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f5ce8b-117e-435b-82c3-456ea7ee27fe admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:37:33.258360 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c589b36f-7d2b-4944-95b8-8b539413fbf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:37:33.259017 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589b36f-7d2b-4944-95b8-8b539413fbf5 service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:37:33.259307 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589b36f-7d2b-4944-95b8-8b539413fbf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:37:33.259588 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589b36f-7d2b-4944-95b8-8b539413fbf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:37:33.261440 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-c589b36f-7d2b-4944-95b8-8b539413fbf5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:37:33.276925 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c589b36f-7d2b-4944-95b8-8b539413fbf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:37:33.278569 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c589b36f-7d2b-4944-95b8-8b539413fbf5 service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:37:33.290771 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1363/2754] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:37:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:37:40.245190 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=38 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:37:43.285411 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=77,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:38:21.234521 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fc4969-fbdb-4121-b9f2-1f0a001f2bcf service cinder] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:38:21.236097 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-c7fc4969-fbdb-4121-b9f2-1f0a001f2bcf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:38:21.251687 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fc4969-fbdb-4121-b9f2-1f0a001f2bcf service cinder] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:38:21.258861 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdb8f7d-fe69-48cf-b235-637ce1371dc9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904255dd3c1f406eb10f6fc65f9cd098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:38:21.259913 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdb8f7d-fe69-48cf-b235-637ce1371dc9 service nova] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:38:21.260212 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdb8f7d-fe69-48cf-b235-637ce1371dc9 service nova] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:38:21.260525 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdb8f7d-fe69-48cf-b235-637ce1371dc9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:38:21.262677 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-dcdb8f7d-fe69-48cf-b235-637ce1371dc9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:38:21.276659 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdb8f7d-fe69-48cf-b235-637ce1371dc9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:38:21.278274 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdb8f7d-fe69-48cf-b235-637ce1371dc9 service nova] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:38:21.292141 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1392/2755] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:38:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:38:26.739863 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac66ab4-1b58-4911-bce9-0a24840cea1a None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:38:26.740434 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac66ab4-1b58-4911-bce9-0a24840cea1a None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:38:26.740553 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac66ab4-1b58-4911-bce9-0a24840cea1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:38:26.762830 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-fac66ab4-1b58-4911-bce9-0a24840cea1a None None] MFA Rules not processed for user `5b424248bfb84fe4a25269232e08dcaf`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:38:26.764255 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-fac66ab4-1b58-4911-bce9-0a24840cea1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:38:26.835305 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1364/2756] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:38:26 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:38:28.630562 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdb8f7d-fe69-48cf-b235-637ce1371dc9 service nova] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:38:28.632223 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-dcdb8f7d-fe69-48cf-b235-637ce1371dc9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:38:28.715108 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdb8f7d-fe69-48cf-b235-637ce1371dc9 service nova] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:38:28.720161 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087fdc9c-08e3-4f9c-a1f3-fd5adee1b014 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938187967efe49be9c5389d1feef81ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:38:28.720915 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087fdc9c-08e3-4f9c-a1f3-fd5adee1b014 service placement] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:38:28.721168 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087fdc9c-08e3-4f9c-a1f3-fd5adee1b014 service placement] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:38:28.721422 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087fdc9c-08e3-4f9c-a1f3-fd5adee1b014 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:38:28.723569 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-087fdc9c-08e3-4f9c-a1f3-fd5adee1b014 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:38:28.736054 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087fdc9c-08e3-4f9c-a1f3-fd5adee1b014 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:38:28.737695 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087fdc9c-08e3-4f9c-a1f3-fd5adee1b014 service placement] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:38:28.748937 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1393/2757] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:38:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:38:34.845449 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac66ab4-1b58-4911-bce9-0a24840cea1a None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:38:34.857039 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac66ab4-1b58-4911-bce9-0a24840cea1a None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:38:34.861486 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b8fa9-4a58-4ea9-8a96-daf2fe158561 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:38:34.862161 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b8fa9-4a58-4ea9-8a96-daf2fe158561 service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:38:34.862452 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b8fa9-4a58-4ea9-8a96-daf2fe158561 service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:38:34.862732 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b8fa9-4a58-4ea9-8a96-daf2fe158561 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:38:34.864786 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-420b8fa9-4a58-4ea9-8a96-daf2fe158561 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:38:34.875324 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420b8fa9-4a58-4ea9-8a96-daf2fe158561 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:38:34.877585 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420b8fa9-4a58-4ea9-8a96-daf2fe158561 service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:38:34.887137 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1365/2758] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:38:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:38:38.743829 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=38 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:38:42.460239 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087fdc9c-08e3-4f9c-a1f3-fd5adee1b014 service placement] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:38:42.462732 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-087fdc9c-08e3-4f9c-a1f3-fd5adee1b014 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:38:42.479475 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087fdc9c-08e3-4f9c-a1f3-fd5adee1b014 service placement] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:38:42.485389 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63aae49-fa1a-4d83-ae88-0b2faa89a676 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904255dd3c1f406eb10f6fc65f9cd098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:38:42.486178 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63aae49-fa1a-4d83-ae88-0b2faa89a676 service nova] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:38:42.486954 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63aae49-fa1a-4d83-ae88-0b2faa89a676 service nova] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:38:42.487872 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63aae49-fa1a-4d83-ae88-0b2faa89a676 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:38:42.489741 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-f63aae49-fa1a-4d83-ae88-0b2faa89a676 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:38:42.508937 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63aae49-fa1a-4d83-ae88-0b2faa89a676 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:38:42.508937 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63aae49-fa1a-4d83-ae88-0b2faa89a676 service nova] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:38:42.520451 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1394/2759] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:38:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:38:44.882651 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=31 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:38:52.515329 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=6 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:39:52.000305 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b8fa9-4a58-4ea9-8a96-daf2fe158561 service neutron] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:39:52.058056 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b8fa9-4a58-4ea9-8a96-daf2fe158561 service neutron] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:39:52.064217 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084260b4-6dec-449e-89fb-8bb1b4b1bf31 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938187967efe49be9c5389d1feef81ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:39:52.064927 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084260b4-6dec-449e-89fb-8bb1b4b1bf31 service placement] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:39:52.065211 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084260b4-6dec-449e-89fb-8bb1b4b1bf31 service placement] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:39:52.065487 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084260b4-6dec-449e-89fb-8bb1b4b1bf31 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:39:52.067283 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-084260b4-6dec-449e-89fb-8bb1b4b1bf31 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:39:52.098916 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084260b4-6dec-449e-89fb-8bb1b4b1bf31 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:39:52.100592 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084260b4-6dec-449e-89fb-8bb1b4b1bf31 service placement] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:39:52.111273 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1366/2760] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:39:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:39:52.146220 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63aae49-fa1a-4d83-ae88-0b2faa89a676 service nova] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:39:52.147932 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-f63aae49-fa1a-4d83-ae88-0b2faa89a676 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:39:52.283329 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63aae49-fa1a-4d83-ae88-0b2faa89a676 service nova] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:39:52.287921 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8008a00-28ef-4efa-8a6a-2d6512bb8977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:39:52.288828 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8008a00-28ef-4efa-8a6a-2d6512bb8977 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:39:52.289398 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8008a00-28ef-4efa-8a6a-2d6512bb8977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:39:52.289714 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8008a00-28ef-4efa-8a6a-2d6512bb8977 admin admin] PATH_INFO: `/v3/users/5b424248bfb84fe4a25269232e08dcaf` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:39:52.290892 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8008a00-28ef-4efa-8a6a-2d6512bb8977 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b424248bfb84fe4a25269232e08dcaf)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:39:52.292845 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8008a00-28ef-4efa-8a6a-2d6512bb8977 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:39:52.486729 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-c8008a00-28ef-4efa-8a6a-2d6512bb8977 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b424248bfb84fe4a25269232e08dcaf'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:39:52.499869 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-c8008a00-28ef-4efa-8a6a-2d6512bb8977 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b424248bfb84fe4a25269232e08dcaf'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:39:52.565602 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-c8008a00-28ef-4efa-8a6a-2d6512bb8977 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b424248bfb84fe4a25269232e08dcaf'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:39:52.566183 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-c8008a00-28ef-4efa-8a6a-2d6512bb8977 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b424248bfb84fe4a25269232e08dcaf'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:39:52.573451 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-c8008a00-28ef-4efa-8a6a-2d6512bb8977 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b424248bfb84fe4a25269232e08dcaf'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:39:52.579664 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1395/2761] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:39:52 2022] DELETE /identity/v3/users/5b424248bfb84fe4a25269232e08dcaf => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:39:52.845067 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084260b4-6dec-449e-89fb-8bb1b4b1bf31 service placement] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:39:52.855951 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084260b4-6dec-449e-89fb-8bb1b4b1bf31 service placement] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:39:52.915942 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7d301b-bcca-4821-9341-c337746f5435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:39:52.916842 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d301b-bcca-4821-9341-c337746f5435 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:39:52.917145 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d301b-bcca-4821-9341-c337746f5435 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:39:52.917409 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d301b-bcca-4821-9341-c337746f5435 admin admin] PATH_INFO: `/v3/projects/466c7ed97ff34cd7824164207ab764d0` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:39:52.918622 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7d301b-bcca-4821-9341-c337746f5435 admin admin] RBAC: Authorizing `identity:delete_project(project_id=466c7ed97ff34cd7824164207ab764d0)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:39:52.920551 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7d301b-bcca-4821-9341-c337746f5435 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:39:53.017669 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-8e7d301b-bcca-4821-9341-c337746f5435 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '466c7ed97ff34cd7824164207ab764d0', 'request_id': 'req-8e7d301b-bcca-4821-9341-c337746f5435'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:39:53.131207 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-8e7d301b-bcca-4821-9341-c337746f5435 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '466c7ed97ff34cd7824164207ab764d0', 'request_id': 'req-8e7d301b-bcca-4821-9341-c337746f5435'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:39:53.137595 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-8e7d301b-bcca-4821-9341-c337746f5435 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '466c7ed97ff34cd7824164207ab764d0', 'request_id': 'req-8e7d301b-bcca-4821-9341-c337746f5435'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:39:53.150170 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-8e7d301b-bcca-4821-9341-c337746f5435 admin admin] The token cache is being invalidate because project 466c7ed97ff34cd7824164207ab764d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:39:53.150443 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-8e7d301b-bcca-4821-9341-c337746f5435 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:39:53.151102 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-8e7d301b-bcca-4821-9341-c337746f5435 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:39:53.152836 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1367/2762] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:39:52 2022] DELETE /identity/v3/projects/466c7ed97ff34cd7824164207ab764d0 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:39:53.158906 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df412a-1532-4f9f-8377-032334c7f39f None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:39:53.159395 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df412a-1532-4f9f-8377-032334c7f39f None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:39:53.159524 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df412a-1532-4f9f-8377-032334c7f39f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:39:53.181427 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:39:53.216884 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-d4df412a-1532-4f9f-8377-032334c7f39f None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:39:53.218912 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-d4df412a-1532-4f9f-8377-032334c7f39f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:39:53.277346 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1396/2763] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:39:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:39:53.285795 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c61868-5b3f-41e0-8776-c33316cd24f0 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:39:53.286088 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c61868-5b3f-41e0-8776-c33316cd24f0 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:39:53.286391 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c61868-5b3f-41e0-8776-c33316cd24f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:39:53.305842 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-91c61868-5b3f-41e0-8776-c33316cd24f0 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:39:53.307106 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-91c61868-5b3f-41e0-8776-c33316cd24f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:39:53.315060 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1368/2764] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:39:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:39:53.319311 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4df412a-1532-4f9f-8377-032334c7f39f None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:39:53.329704 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4df412a-1532-4f9f-8377-032334c7f39f None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:39:53.334387 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7b8a34-d27c-41c4-ba19-90074e1beb2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:39:53.335041 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b8a34-d27c-41c4-ba19-90074e1beb2a admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:39:53.335361 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b8a34-d27c-41c4-ba19-90074e1beb2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:39:53.335752 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b8a34-d27c-41c4-ba19-90074e1beb2a admin admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:39:53.336529 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7b8a34-d27c-41c4-ba19-90074e1beb2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:39:53.336835 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7b8a34-d27c-41c4-ba19-90074e1beb2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:39:53.338431 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7b8a34-d27c-41c4-ba19-90074e1beb2a admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:39:53.351300 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1397/2765] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:39:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:39:53.355620 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c61868-5b3f-41e0-8776-c33316cd24f0 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:39:53.365118 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c61868-5b3f-41e0-8776-c33316cd24f0 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:39:53.369833 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c04739-e6e8-4537-87a2-e9d88c8683fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:39:53.370483 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c04739-e6e8-4537-87a2-e9d88c8683fa admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:39:53.370765 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c04739-e6e8-4537-87a2-e9d88c8683fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:39:53.371043 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c04739-e6e8-4537-87a2-e9d88c8683fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:39:53.371751 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c04739-e6e8-4537-87a2-e9d88c8683fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:39:53.373662 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c04739-e6e8-4537-87a2-e9d88c8683fa admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:39:53.481645 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:39:53.488161 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1369/2766] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:39:53 2022] POST /identity/v3/projects => generated 369 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:39:53.495079 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7b8a34-d27c-41c4-ba19-90074e1beb2a admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:39:53.503859 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7b8a34-d27c-41c4-ba19-90074e1beb2a admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:39:53.560690 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f036d-78c9-47cd-9024-c5a007126d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:39:53.561397 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f036d-78c9-47cd-9024-c5a007126d74 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:39:53.561686 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f036d-78c9-47cd-9024-c5a007126d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:39:53.562496 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f036d-78c9-47cd-9024-c5a007126d74 admin admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:39:53.563224 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525f036d-78c9-47cd-9024-c5a007126d74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:39:53.565311 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525f036d-78c9-47cd-9024-c5a007126d74 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:39:53.686035 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1398/2767] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:39:53 2022] POST /identity/v3/users => generated 285 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:39:53.690851 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c04739-e6e8-4537-87a2-e9d88c8683fa admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:39:53.701615 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c04739-e6e8-4537-87a2-e9d88c8683fa admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:39:53.706843 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a71875a-f5f3-4b52-8864-3281e1e06776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:39:53.707506 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71875a-f5f3-4b52-8864-3281e1e06776 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:39:53.707881 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71875a-f5f3-4b52-8864-3281e1e06776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:39:53.708089 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71875a-f5f3-4b52-8864-3281e1e06776 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:39:53.708861 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a71875a-f5f3-4b52-8864-3281e1e06776 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:39:53.710461 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a71875a-f5f3-4b52-8864-3281e1e06776 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:39:53.721302 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1370/2768] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:39:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:39:53.725550 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f036d-78c9-47cd-9024-c5a007126d74 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:39:53.736194 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f036d-78c9-47cd-9024-c5a007126d74 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:39:53.741037 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ac6c8-bb9a-49ce-8663-9a61b8659974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:39:53.741809 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ac6c8-bb9a-49ce-8663-9a61b8659974 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:39:53.742094 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ac6c8-bb9a-49ce-8663-9a61b8659974 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:39:53.742369 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ac6c8-bb9a-49ce-8663-9a61b8659974 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:39:53.743094 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2ac6c8-bb9a-49ce-8663-9a61b8659974 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:39:53.744712 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2ac6c8-bb9a-49ce-8663-9a61b8659974 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:39:53.755737 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1399/2769] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:39:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:39:53.760221 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a71875a-f5f3-4b52-8864-3281e1e06776 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:39:53.771121 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a71875a-f5f3-4b52-8864-3281e1e06776 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:39:53.775866 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a0a135-0619-454a-bf8d-dfd5498b9e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:39:53.776790 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a0a135-0619-454a-bf8d-dfd5498b9e7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:39:53.777078 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a0a135-0619-454a-bf8d-dfd5498b9e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:39:53.777388 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a0a135-0619-454a-bf8d-dfd5498b9e7a admin admin] PATH_INFO: `/v3/projects/cbf331c0e8d3451292cdf27b54256dcb/users/c7dc8cc420314bafba208bd4a39f6c40/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:39:53.792006 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a0a135-0619-454a-bf8d-dfd5498b9e7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbf331c0e8d3451292cdf27b54256dcb, user_id=c7dc8cc420314bafba208bd4a39f6c40, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:39:53.794683 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a0a135-0619-454a-bf8d-dfd5498b9e7a admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:39:53.874990 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1371/2770] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:39:53 2022] PUT /identity/v3/projects/cbf331c0e8d3451292cdf27b54256dcb/users/c7dc8cc420314bafba208bd4a39f6c40/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:39:53.886133 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c21747-7bbb-4c20-a24c-34e3d19dc854 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:39:53.888990 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c21747-7bbb-4c20-a24c-34e3d19dc854 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:39:53.888990 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c21747-7bbb-4c20-a24c-34e3d19dc854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:39:53.913090 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-11c21747-7bbb-4c20-a24c-34e3d19dc854 None None] MFA Rules not processed for user `c7dc8cc420314bafba208bd4a39f6c40`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:39:53.914289 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-11c21747-7bbb-4c20-a24c-34e3d19dc854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:39:53.958454 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:39:53.981671 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1400/2771] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:39:53 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:39:53.986701 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a0a135-0619-454a-bf8d-dfd5498b9e7a admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:39:54.052249 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a0a135-0619-454a-bf8d-dfd5498b9e7a admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:39:54.056936 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27128dbc-0812-438e-86e0-1aff81c66758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:39:54.057591 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27128dbc-0812-438e-86e0-1aff81c66758 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:39:54.057875 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27128dbc-0812-438e-86e0-1aff81c66758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:39:54.058152 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27128dbc-0812-438e-86e0-1aff81c66758 admin admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:39:54.058832 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27128dbc-0812-438e-86e0-1aff81c66758 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:39:54.060752 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27128dbc-0812-438e-86e0-1aff81c66758 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:39:54.167369 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1372/2772] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:39:53 2022] POST /identity/v3/projects => generated 369 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:39:54.171552 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c21747-7bbb-4c20-a24c-34e3d19dc854 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:39:54.183240 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c21747-7bbb-4c20-a24c-34e3d19dc854 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:39:54.240571 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b74aad7-6dfc-4fdb-a310-f02ef92ca5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:39:54.241156 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b74aad7-6dfc-4fdb-a310-f02ef92ca5ef admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:39:54.241446 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b74aad7-6dfc-4fdb-a310-f02ef92ca5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:39:54.241724 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b74aad7-6dfc-4fdb-a310-f02ef92ca5ef admin admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:39:54.242420 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b74aad7-6dfc-4fdb-a310-f02ef92ca5ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:39:54.244332 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b74aad7-6dfc-4fdb-a310-f02ef92ca5ef admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:39:54.337700 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1401/2773] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:39:54 2022] POST /identity/v3/users => generated 291 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:39:54.342032 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27128dbc-0812-438e-86e0-1aff81c66758 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:39:54.353804 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27128dbc-0812-438e-86e0-1aff81c66758 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:39:54.358643 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12681d08-225e-485b-b041-440ffcd42663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:39:54.359329 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12681d08-225e-485b-b041-440ffcd42663 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:39:54.359623 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12681d08-225e-485b-b041-440ffcd42663 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:39:54.359901 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12681d08-225e-485b-b041-440ffcd42663 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:39:54.361449 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12681d08-225e-485b-b041-440ffcd42663 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:39:54.362990 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12681d08-225e-485b-b041-440ffcd42663 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:39:54.373754 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1373/2774] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:39:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:39:54.378033 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b74aad7-6dfc-4fdb-a310-f02ef92ca5ef admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:39:54.388753 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b74aad7-6dfc-4fdb-a310-f02ef92ca5ef admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:39:54.393646 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c841c41-9b21-4497-8496-7a0ba09b6951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:39:54.394531 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c841c41-9b21-4497-8496-7a0ba09b6951 admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:39:54.394817 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c841c41-9b21-4497-8496-7a0ba09b6951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:39:54.395097 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c841c41-9b21-4497-8496-7a0ba09b6951 admin admin] PATH_INFO: `/v3/projects/3a8cdde3c06f44a2a128aa4aa48630f4/users/a12e040876ab40aea1c0ccca8d36d5b6/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:39:54.410770 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c841c41-9b21-4497-8496-7a0ba09b6951 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a8cdde3c06f44a2a128aa4aa48630f4, user_id=a12e040876ab40aea1c0ccca8d36d5b6, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:39:54.413446 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c841c41-9b21-4497-8496-7a0ba09b6951 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:39:54.492015 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1402/2775] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:39:54 2022] PUT /identity/v3/projects/3a8cdde3c06f44a2a128aa4aa48630f4/users/a12e040876ab40aea1c0ccca8d36d5b6/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:39:54.495848 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12681d08-225e-485b-b041-440ffcd42663 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:39:54.506896 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12681d08-225e-485b-b041-440ffcd42663 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:39:54.568297 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ceddb7-707b-4256-a1f4-b6bbd53fd308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:39:54.569004 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ceddb7-707b-4256-a1f4-b6bbd53fd308 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:39:54.569314 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ceddb7-707b-4256-a1f4-b6bbd53fd308 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:39:54.569594 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ceddb7-707b-4256-a1f4-b6bbd53fd308 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:39:54.570316 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ceddb7-707b-4256-a1f4-b6bbd53fd308 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:39:54.571922 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ceddb7-707b-4256-a1f4-b6bbd53fd308 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:39:54.582597 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1374/2776] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:39:54 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:39:54.586847 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c841c41-9b21-4497-8496-7a0ba09b6951 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:39:54.598591 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c841c41-9b21-4497-8496-7a0ba09b6951 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:39:54.603659 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9372300b-d86e-4de4-9cbb-09226de98dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:39:54.604973 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372300b-d86e-4de4-9cbb-09226de98dbb admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:39:54.604973 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372300b-d86e-4de4-9cbb-09226de98dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:39:54.605350 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372300b-d86e-4de4-9cbb-09226de98dbb admin admin] PATH_INFO: `/v3/projects/3a8cdde3c06f44a2a128aa4aa48630f4/users/a12e040876ab40aea1c0ccca8d36d5b6/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:39:54.607457 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9372300b-d86e-4de4-9cbb-09226de98dbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a8cdde3c06f44a2a128aa4aa48630f4, user_id=a12e040876ab40aea1c0ccca8d36d5b6, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:39:54.610249 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9372300b-d86e-4de4-9cbb-09226de98dbb admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:39:54.621204 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1403/2777] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:39:54 2022] PUT /identity/v3/projects/3a8cdde3c06f44a2a128aa4aa48630f4/users/a12e040876ab40aea1c0ccca8d36d5b6/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:39:54.638249 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f37ed4-ab91-4e2a-bb2f-7dd8e506be00 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:39:54.638539 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f37ed4-ab91-4e2a-bb2f-7dd8e506be00 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:39:54.638824 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f37ed4-ab91-4e2a-bb2f-7dd8e506be00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:39:54.668617 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-21f37ed4-ab91-4e2a-bb2f-7dd8e506be00 None None] MFA Rules not processed for user `a12e040876ab40aea1c0ccca8d36d5b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:39:54.669826 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-21f37ed4-ab91-4e2a-bb2f-7dd8e506be00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:39:54.695034 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:39:54.741714 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1375/2778] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:39:54 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:39:54.752892 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9372300b-d86e-4de4-9cbb-09226de98dbb admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:39:54.754687 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-9372300b-d86e-4de4-9cbb-09226de98dbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:39:54.836825 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9372300b-d86e-4de4-9cbb-09226de98dbb admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:39:54.840785 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d12f01-623f-497f-85c2-8f4598d2cefd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:39:54.841430 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d12f01-623f-497f-85c2-8f4598d2cefd service cinder] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:39:54.841714 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d12f01-623f-497f-85c2-8f4598d2cefd service cinder] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:39:54.841994 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d12f01-623f-497f-85c2-8f4598d2cefd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:39:54.852209 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d12f01-623f-497f-85c2-8f4598d2cefd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:39:54.853803 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d12f01-623f-497f-85c2-8f4598d2cefd service cinder] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:39:54.864333 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1404/2779] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:39:54 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:39:54.986762 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f37ed4-ab91-4e2a-bb2f-7dd8e506be00 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:39:54.989110 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-21f37ed4-ab91-4e2a-bb2f-7dd8e506be00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:39:55.000948 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f37ed4-ab91-4e2a-bb2f-7dd8e506be00 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:39:55.004489 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a060277-1c45-4952-a87d-bc3cb98fe031 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:39:55.005170 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a060277-1c45-4952-a87d-bc3cb98fe031 service cinder] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:39:55.005459 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a060277-1c45-4952-a87d-bc3cb98fe031 service cinder] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:39:55.005738 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a060277-1c45-4952-a87d-bc3cb98fe031 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:39:55.062330 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a060277-1c45-4952-a87d-bc3cb98fe031 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:39:55.063826 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a060277-1c45-4952-a87d-bc3cb98fe031 service cinder] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:39:55.072811 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1376/2780] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:39:54 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:40:00.243722 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d12f01-623f-497f-85c2-8f4598d2cefd service cinder] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:00.265764 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:40:00.308198 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d12f01-623f-497f-85c2-8f4598d2cefd service cinder] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:00.312836 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e239ff27-e632-46a7-8391-1a538095c32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:00.313700 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e239ff27-e632-46a7-8391-1a538095c32a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:00.313983 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e239ff27-e632-46a7-8391-1a538095c32a admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:00.314259 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e239ff27-e632-46a7-8391-1a538095c32a admin admin] PATH_INFO: `/v3/users/c7dc8cc420314bafba208bd4a39f6c40` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:00.315480 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e239ff27-e632-46a7-8391-1a538095c32a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7dc8cc420314bafba208bd4a39f6c40)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:00.317506 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e239ff27-e632-46a7-8391-1a538095c32a admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:00.476716 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-e239ff27-e632-46a7-8391-1a538095c32a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7dc8cc420314bafba208bd4a39f6c40'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:00.489194 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-e239ff27-e632-46a7-8391-1a538095c32a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7dc8cc420314bafba208bd4a39f6c40'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:00.568372 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-e239ff27-e632-46a7-8391-1a538095c32a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7dc8cc420314bafba208bd4a39f6c40'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:00.569034 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-e239ff27-e632-46a7-8391-1a538095c32a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7dc8cc420314bafba208bd4a39f6c40'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:00.576791 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-e239ff27-e632-46a7-8391-1a538095c32a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7dc8cc420314bafba208bd4a39f6c40'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:00.583464 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1405/2781] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:40:00 2022] DELETE /identity/v3/users/c7dc8cc420314bafba208bd4a39f6c40 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:40:00.588199 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe6c87-c1d2-4680-b1bf-a2365089787e None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:00.588713 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe6c87-c1d2-4680-b1bf-a2365089787e None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:00.588850 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe6c87-c1d2-4680-b1bf-a2365089787e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:00.607015 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-dbbe6c87-c1d2-4680-b1bf-a2365089787e None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:40:00.608223 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-dbbe6c87-c1d2-4680-b1bf-a2365089787e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:00.667661 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1377/2782] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:40:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:40:00.677361 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e239ff27-e632-46a7-8391-1a538095c32a admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:00.678971 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-e239ff27-e632-46a7-8391-1a538095c32a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:00.758270 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e239ff27-e632-46a7-8391-1a538095c32a admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:00.762171 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be944b2-9fc5-4ead-8b4f-a8c931ccb22c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:00.762829 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be944b2-9fc5-4ead-8b4f-a8c931ccb22c service neutron] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:00.763112 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be944b2-9fc5-4ead-8b4f-a8c931ccb22c service neutron] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:00.763395 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be944b2-9fc5-4ead-8b4f-a8c931ccb22c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:00.773124 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be944b2-9fc5-4ead-8b4f-a8c931ccb22c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:00.774547 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be944b2-9fc5-4ead-8b4f-a8c931ccb22c service neutron] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:00.783937 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1406/2783] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:40:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:40:01.199176 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbe6c87-c1d2-4680-b1bf-a2365089787e None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:01.199176 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-dbbe6c87-c1d2-4680-b1bf-a2365089787e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:01.214261 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbe6c87-c1d2-4680-b1bf-a2365089787e None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:01.218970 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e60bf4d-6453-4616-bc72-45d9877a56c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:01.219847 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e60bf4d-6453-4616-bc72-45d9877a56c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:01.220124 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e60bf4d-6453-4616-bc72-45d9877a56c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:01.220424 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e60bf4d-6453-4616-bc72-45d9877a56c0 admin admin] PATH_INFO: `/v3/projects/cbf331c0e8d3451292cdf27b54256dcb` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:01.221713 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e60bf4d-6453-4616-bc72-45d9877a56c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbf331c0e8d3451292cdf27b54256dcb)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:01.224535 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e60bf4d-6453-4616-bc72-45d9877a56c0 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:01.343369 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-6e60bf4d-6453-4616-bc72-45d9877a56c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbf331c0e8d3451292cdf27b54256dcb', 'request_id': 'req-6e60bf4d-6453-4616-bc72-45d9877a56c0'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:01.346904 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:40:01.449406 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-6e60bf4d-6453-4616-bc72-45d9877a56c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbf331c0e8d3451292cdf27b54256dcb', 'request_id': 'req-6e60bf4d-6453-4616-bc72-45d9877a56c0'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:01.455508 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-6e60bf4d-6453-4616-bc72-45d9877a56c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbf331c0e8d3451292cdf27b54256dcb', 'request_id': 'req-6e60bf4d-6453-4616-bc72-45d9877a56c0'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:01.467239 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-6e60bf4d-6453-4616-bc72-45d9877a56c0 admin admin] The token cache is being invalidate because project cbf331c0e8d3451292cdf27b54256dcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:40:01.467536 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-6e60bf4d-6453-4616-bc72-45d9877a56c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:01.468188 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-6e60bf4d-6453-4616-bc72-45d9877a56c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:01.469848 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1378/2784] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:40:01 2022] DELETE /identity/v3/projects/cbf331c0e8d3451292cdf27b54256dcb => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:40:01.473981 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be944b2-9fc5-4ead-8b4f-a8c931ccb22c service neutron] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:01.483803 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be944b2-9fc5-4ead-8b4f-a8c931ccb22c service neutron] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:01.500652 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:40:01.541496 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02a776c-0957-4077-bde3-6bd5ad7d3259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:01.542369 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a776c-0957-4077-bde3-6bd5ad7d3259 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:01.542659 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a776c-0957-4077-bde3-6bd5ad7d3259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:01.542937 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a776c-0957-4077-bde3-6bd5ad7d3259 admin admin] PATH_INFO: `/v3/users/a12e040876ab40aea1c0ccca8d36d5b6` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:01.544138 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02a776c-0957-4077-bde3-6bd5ad7d3259 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a12e040876ab40aea1c0ccca8d36d5b6)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:01.546086 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02a776c-0957-4077-bde3-6bd5ad7d3259 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:01.743656 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-e02a776c-0957-4077-bde3-6bd5ad7d3259 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a12e040876ab40aea1c0ccca8d36d5b6'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:01.756249 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-e02a776c-0957-4077-bde3-6bd5ad7d3259 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a12e040876ab40aea1c0ccca8d36d5b6'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:01.825865 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-e02a776c-0957-4077-bde3-6bd5ad7d3259 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a12e040876ab40aea1c0ccca8d36d5b6'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:01.826494 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-e02a776c-0957-4077-bde3-6bd5ad7d3259 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a12e040876ab40aea1c0ccca8d36d5b6'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:01.833988 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-e02a776c-0957-4077-bde3-6bd5ad7d3259 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a12e040876ab40aea1c0ccca8d36d5b6'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:01.839932 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1407/2785] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:40:01 2022] DELETE /identity/v3/users/a12e040876ab40aea1c0ccca8d36d5b6 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:40:02.312958 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e60bf4d-6453-4616-bc72-45d9877a56c0 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:02.317563 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-6e60bf4d-6453-4616-bc72-45d9877a56c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:02.327637 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e60bf4d-6453-4616-bc72-45d9877a56c0 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:02.387617 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a47735-5cf5-41a1-99d0-4a6717e36330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:02.388517 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a47735-5cf5-41a1-99d0-4a6717e36330 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:02.388833 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a47735-5cf5-41a1-99d0-4a6717e36330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:02.389379 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a47735-5cf5-41a1-99d0-4a6717e36330 admin admin] PATH_INFO: `/v3/projects/3a8cdde3c06f44a2a128aa4aa48630f4` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:02.390566 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a47735-5cf5-41a1-99d0-4a6717e36330 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a8cdde3c06f44a2a128aa4aa48630f4)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:02.392487 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a47735-5cf5-41a1-99d0-4a6717e36330 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:02.510721 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-70a47735-5cf5-41a1-99d0-4a6717e36330 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a8cdde3c06f44a2a128aa4aa48630f4', 'request_id': 'req-70a47735-5cf5-41a1-99d0-4a6717e36330'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:02.565984 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-70a47735-5cf5-41a1-99d0-4a6717e36330 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a8cdde3c06f44a2a128aa4aa48630f4', 'request_id': 'req-70a47735-5cf5-41a1-99d0-4a6717e36330'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:02.573143 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-70a47735-5cf5-41a1-99d0-4a6717e36330 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a8cdde3c06f44a2a128aa4aa48630f4', 'request_id': 'req-70a47735-5cf5-41a1-99d0-4a6717e36330'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:02.586100 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-70a47735-5cf5-41a1-99d0-4a6717e36330 admin admin] The token cache is being invalidate because project 3a8cdde3c06f44a2a128aa4aa48630f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:40:02.586393 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-70a47735-5cf5-41a1-99d0-4a6717e36330 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:02.587099 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-70a47735-5cf5-41a1-99d0-4a6717e36330 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:02.588728 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1379/2786] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:40:02 2022] DELETE /identity/v3/projects/3a8cdde3c06f44a2a128aa4aa48630f4 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:40:02.593799 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1037620c-7f57-41a7-915a-15e85e356d62 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:02.594299 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1037620c-7f57-41a7-915a-15e85e356d62 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:02.594406 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1037620c-7f57-41a7-915a-15e85e356d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:02.612504 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-1037620c-7f57-41a7-915a-15e85e356d62 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:40:02.613867 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-1037620c-7f57-41a7-915a-15e85e356d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:02.676283 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1408/2787] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:40:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:40:02.684099 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf1068-50fd-488c-a831-a050ff6beab2 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:02.684414 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf1068-50fd-488c-a831-a050ff6beab2 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:02.684677 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf1068-50fd-488c-a831-a050ff6beab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:02.703308 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-78bf1068-50fd-488c-a831-a050ff6beab2 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:40:02.704543 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-78bf1068-50fd-488c-a831-a050ff6beab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:02.713374 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1380/2788] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:40:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:40:02.717644 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1037620c-7f57-41a7-915a-15e85e356d62 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:02.728071 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1037620c-7f57-41a7-915a-15e85e356d62 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:02.732998 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214fa566-c9cf-4afa-a8d6-81d3a376c20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:02.733595 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214fa566-c9cf-4afa-a8d6-81d3a376c20f admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:02.733874 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214fa566-c9cf-4afa-a8d6-81d3a376c20f admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:02.734264 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214fa566-c9cf-4afa-a8d6-81d3a376c20f admin admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:02.735034 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214fa566-c9cf-4afa-a8d6-81d3a376c20f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:40:02.735342 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214fa566-c9cf-4afa-a8d6-81d3a376c20f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:02.737089 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214fa566-c9cf-4afa-a8d6-81d3a376c20f admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:02.750529 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1409/2789] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:40:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:40:02.755548 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bf1068-50fd-488c-a831-a050ff6beab2 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:02.765883 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bf1068-50fd-488c-a831-a050ff6beab2 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:02.770688 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0292ed63-29cf-4550-ba10-563821672bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:02.771340 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0292ed63-29cf-4550-ba10-563821672bd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:02.771628 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0292ed63-29cf-4550-ba10-563821672bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:02.771915 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0292ed63-29cf-4550-ba10-563821672bd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:02.772650 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0292ed63-29cf-4550-ba10-563821672bd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:02.774698 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0292ed63-29cf-4550-ba10-563821672bd9 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:02.919217 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1381/2790] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:40:02 2022] POST /identity/v3/projects => generated 385 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:40:02.923563 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214fa566-c9cf-4afa-a8d6-81d3a376c20f admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:02.934541 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214fa566-c9cf-4afa-a8d6-81d3a376c20f admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:02.959281 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:40:02.993660 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42465abd-c3bd-4f49-b445-c18fc99e7fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:02.994333 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42465abd-c3bd-4f49-b445-c18fc99e7fe2 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:02.994638 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42465abd-c3bd-4f49-b445-c18fc99e7fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:02.995023 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42465abd-c3bd-4f49-b445-c18fc99e7fe2 admin admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:02.995634 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42465abd-c3bd-4f49-b445-c18fc99e7fe2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:02.997575 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42465abd-c3bd-4f49-b445-c18fc99e7fe2 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:03.083732 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1410/2791] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:40:02 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:40:03.087927 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0292ed63-29cf-4550-ba10-563821672bd9 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:03.098975 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0292ed63-29cf-4550-ba10-563821672bd9 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:03.103874 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053158a3-586c-483f-9450-feecd5ed5631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:03.104577 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053158a3-586c-483f-9450-feecd5ed5631 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:03.104868 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053158a3-586c-483f-9450-feecd5ed5631 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:03.105149 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053158a3-586c-483f-9450-feecd5ed5631 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:03.105880 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053158a3-586c-483f-9450-feecd5ed5631 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:03.107484 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053158a3-586c-483f-9450-feecd5ed5631 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:03.118731 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1382/2792] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:40:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:40:03.122682 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42465abd-c3bd-4f49-b445-c18fc99e7fe2 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:03.133261 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42465abd-c3bd-4f49-b445-c18fc99e7fe2 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:03.138259 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bd7a02-8b90-4702-8533-b623892d5edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:03.138919 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd7a02-8b90-4702-8533-b623892d5edd admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:03.139202 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd7a02-8b90-4702-8533-b623892d5edd admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:03.139504 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd7a02-8b90-4702-8533-b623892d5edd admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:03.140231 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bd7a02-8b90-4702-8533-b623892d5edd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:03.141831 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bd7a02-8b90-4702-8533-b623892d5edd admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:03.151616 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1411/2793] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:40:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:40:03.155890 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053158a3-586c-483f-9450-feecd5ed5631 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:03.166492 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053158a3-586c-483f-9450-feecd5ed5631 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:03.171078 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a20a732-5818-43ec-b427-09cad88ae32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:03.171954 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a20a732-5818-43ec-b427-09cad88ae32a admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:03.172253 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a20a732-5818-43ec-b427-09cad88ae32a admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:03.172549 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a20a732-5818-43ec-b427-09cad88ae32a admin admin] PATH_INFO: `/v3/projects/1a0a7a2c316c418d853e83caa358832e/users/98e9149eacf344d6bf5799899ac34c88/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:03.187809 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a20a732-5818-43ec-b427-09cad88ae32a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a0a7a2c316c418d853e83caa358832e, user_id=98e9149eacf344d6bf5799899ac34c88, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:03.190471 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a20a732-5818-43ec-b427-09cad88ae32a admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:03.266090 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1383/2794] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:40:03 2022] PUT /identity/v3/projects/1a0a7a2c316c418d853e83caa358832e/users/98e9149eacf344d6bf5799899ac34c88/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:40:03.276737 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b086cb17-a267-4ff6-ab1e-55a8617a5435 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:03.279397 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b086cb17-a267-4ff6-ab1e-55a8617a5435 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:03.279397 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b086cb17-a267-4ff6-ab1e-55a8617a5435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:03.306141 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-b086cb17-a267-4ff6-ab1e-55a8617a5435 None None] MFA Rules not processed for user `98e9149eacf344d6bf5799899ac34c88`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:40:03.307349 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-b086cb17-a267-4ff6-ab1e-55a8617a5435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:03.371680 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1412/2795] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:40:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:40:03.376094 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a20a732-5818-43ec-b427-09cad88ae32a admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:03.387349 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:40:03.438677 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a20a732-5818-43ec-b427-09cad88ae32a admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:03.443421 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18b8174-269b-463d-8a91-c2ebf7ae348e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:03.444249 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18b8174-269b-463d-8a91-c2ebf7ae348e admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:03.448934 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18b8174-269b-463d-8a91-c2ebf7ae348e admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:03.448934 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18b8174-269b-463d-8a91-c2ebf7ae348e admin admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:03.448934 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18b8174-269b-463d-8a91-c2ebf7ae348e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:03.448934 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18b8174-269b-463d-8a91-c2ebf7ae348e admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:03.528316 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1384/2796] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:40:03 2022] POST /identity/v3/projects => generated 387 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:40:03.532369 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b086cb17-a267-4ff6-ab1e-55a8617a5435 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:03.544347 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b086cb17-a267-4ff6-ab1e-55a8617a5435 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:03.602727 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de0f654-32a5-4064-8cc3-d76eaf09fff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:03.603398 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de0f654-32a5-4064-8cc3-d76eaf09fff7 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:03.603676 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de0f654-32a5-4064-8cc3-d76eaf09fff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:03.603951 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de0f654-32a5-4064-8cc3-d76eaf09fff7 admin admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:03.604699 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de0f654-32a5-4064-8cc3-d76eaf09fff7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:03.606638 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de0f654-32a5-4064-8cc3-d76eaf09fff7 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:03.726870 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1413/2797] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:40:03 2022] POST /identity/v3/users => generated 300 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:40:03.731226 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18b8174-269b-463d-8a91-c2ebf7ae348e admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:03.741889 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18b8174-269b-463d-8a91-c2ebf7ae348e admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:03.746556 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c33945-8d0c-41c9-8e92-d10bbadd901c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:03.747222 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c33945-8d0c-41c9-8e92-d10bbadd901c admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:03.747510 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c33945-8d0c-41c9-8e92-d10bbadd901c admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:03.747789 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c33945-8d0c-41c9-8e92-d10bbadd901c admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:03.748556 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c33945-8d0c-41c9-8e92-d10bbadd901c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:03.750172 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c33945-8d0c-41c9-8e92-d10bbadd901c admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:03.761251 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1385/2798] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:40:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:40:03.765660 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de0f654-32a5-4064-8cc3-d76eaf09fff7 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:03.776672 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de0f654-32a5-4064-8cc3-d76eaf09fff7 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:03.781689 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c746c212-ad0a-4853-95d5-66604f1dbf41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:03.782619 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c746c212-ad0a-4853-95d5-66604f1dbf41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:03.782905 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c746c212-ad0a-4853-95d5-66604f1dbf41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:03.783191 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c746c212-ad0a-4853-95d5-66604f1dbf41 admin admin] PATH_INFO: `/v3/projects/8f5c39d5bdc948258e7f62dedb242138/users/6c689d3168814997a2104c9e7ecffa54/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:03.798613 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c746c212-ad0a-4853-95d5-66604f1dbf41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f5c39d5bdc948258e7f62dedb242138, user_id=6c689d3168814997a2104c9e7ecffa54, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:03.801300 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c746c212-ad0a-4853-95d5-66604f1dbf41 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:03.900031 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:40:03.904959 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1414/2799] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:40:03 2022] PUT /identity/v3/projects/8f5c39d5bdc948258e7f62dedb242138/users/6c689d3168814997a2104c9e7ecffa54/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:40:03.910006 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c33945-8d0c-41c9-8e92-d10bbadd901c admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:03.919736 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c33945-8d0c-41c9-8e92-d10bbadd901c admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:03.979659 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ce077e-b1dd-4595-a2aa-e0863783adc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:03.980818 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ce077e-b1dd-4595-a2aa-e0863783adc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:03.980818 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ce077e-b1dd-4595-a2aa-e0863783adc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:03.980818 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ce077e-b1dd-4595-a2aa-e0863783adc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:03.981628 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ce077e-b1dd-4595-a2aa-e0863783adc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:03.983251 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ce077e-b1dd-4595-a2aa-e0863783adc8 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:03.993320 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1386/2800] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:40:03 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:40:03.997546 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c746c212-ad0a-4853-95d5-66604f1dbf41 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:04.009300 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c746c212-ad0a-4853-95d5-66604f1dbf41 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:04.013896 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c482f164-f8d7-4650-af20-3745ffd3c083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:04.014786 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c482f164-f8d7-4650-af20-3745ffd3c083 admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:04.015067 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c482f164-f8d7-4650-af20-3745ffd3c083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:04.015340 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c482f164-f8d7-4650-af20-3745ffd3c083 admin admin] PATH_INFO: `/v3/projects/8f5c39d5bdc948258e7f62dedb242138/users/6c689d3168814997a2104c9e7ecffa54/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:04.017334 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c482f164-f8d7-4650-af20-3745ffd3c083 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f5c39d5bdc948258e7f62dedb242138, user_id=6c689d3168814997a2104c9e7ecffa54, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:04.019938 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c482f164-f8d7-4650-af20-3745ffd3c083 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:04.031528 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1415/2801] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:40:03 2022] PUT /identity/v3/projects/8f5c39d5bdc948258e7f62dedb242138/users/6c689d3168814997a2104c9e7ecffa54/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:40:04.043305 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99fe142-8cfb-4660-be5f-3caa62de8852 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:04.043592 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99fe142-8cfb-4660-be5f-3caa62de8852 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:04.043868 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99fe142-8cfb-4660-be5f-3caa62de8852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:04.071462 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-c99fe142-8cfb-4660-be5f-3caa62de8852 None None] MFA Rules not processed for user `6c689d3168814997a2104c9e7ecffa54`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:40:04.072772 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-c99fe142-8cfb-4660-be5f-3caa62de8852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:04.143325 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1387/2802] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:40:04 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:40:04.155057 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c482f164-f8d7-4650-af20-3745ffd3c083 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:04.156748 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-c482f164-f8d7-4650-af20-3745ffd3c083 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:04.214354 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c482f164-f8d7-4650-af20-3745ffd3c083 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:04.218176 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b1e09b-4f0d-4980-8f9e-7129e746ad2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:04.218829 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1e09b-4f0d-4980-8f9e-7129e746ad2c service cinder] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:04.219108 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1e09b-4f0d-4980-8f9e-7129e746ad2c service cinder] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:04.219385 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1e09b-4f0d-4980-8f9e-7129e746ad2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:04.229395 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b1e09b-4f0d-4980-8f9e-7129e746ad2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:04.230882 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b1e09b-4f0d-4980-8f9e-7129e746ad2c service cinder] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:04.240352 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1416/2803] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:40:04 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:40:04.304716 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99fe142-8cfb-4660-be5f-3caa62de8852 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:04.310267 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:40:04.367297 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99fe142-8cfb-4660-be5f-3caa62de8852 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:04.372062 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebb25ae-4865-484a-9ca4-1ab73342f7e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:04.372985 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb25ae-4865-484a-9ca4-1ab73342f7e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:04.373664 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb25ae-4865-484a-9ca4-1ab73342f7e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:04.374187 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb25ae-4865-484a-9ca4-1ab73342f7e7 admin admin] PATH_INFO: `/v3/users/98e9149eacf344d6bf5799899ac34c88` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:04.375882 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebb25ae-4865-484a-9ca4-1ab73342f7e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98e9149eacf344d6bf5799899ac34c88)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:04.377920 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebb25ae-4865-484a-9ca4-1ab73342f7e7 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:04.600668 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-cebb25ae-4865-484a-9ca4-1ab73342f7e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98e9149eacf344d6bf5799899ac34c88'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:04.612547 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-cebb25ae-4865-484a-9ca4-1ab73342f7e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98e9149eacf344d6bf5799899ac34c88'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:04.613258 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-cebb25ae-4865-484a-9ca4-1ab73342f7e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98e9149eacf344d6bf5799899ac34c88'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:04.620332 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-cebb25ae-4865-484a-9ca4-1ab73342f7e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98e9149eacf344d6bf5799899ac34c88'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:04.620997 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-cebb25ae-4865-484a-9ca4-1ab73342f7e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98e9149eacf344d6bf5799899ac34c88'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:04.712506 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1388/2804] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:40:04 2022] DELETE /identity/v3/users/98e9149eacf344d6bf5799899ac34c88 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:40:04.717181 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73270bf6-9eac-4c54-afc8-bcb24eade858 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:04.717697 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73270bf6-9eac-4c54-afc8-bcb24eade858 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:04.717832 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73270bf6-9eac-4c54-afc8-bcb24eade858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:04.735352 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-73270bf6-9eac-4c54-afc8-bcb24eade858 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:40:04.736595 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-73270bf6-9eac-4c54-afc8-bcb24eade858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:04.794884 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1417/2805] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:40:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:40:04.806258 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebb25ae-4865-484a-9ca4-1ab73342f7e7 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:04.807705 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-cebb25ae-4865-484a-9ca4-1ab73342f7e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:04.867092 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebb25ae-4865-484a-9ca4-1ab73342f7e7 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:04.870946 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc977eb-4305-4624-8639-5cc46134a61c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:04.871611 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc977eb-4305-4624-8639-5cc46134a61c service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:04.872274 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc977eb-4305-4624-8639-5cc46134a61c service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:04.872274 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc977eb-4305-4624-8639-5cc46134a61c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:04.882213 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc977eb-4305-4624-8639-5cc46134a61c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:04.884038 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc977eb-4305-4624-8639-5cc46134a61c service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:04.894339 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1389/2806] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:40:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:40:05.327990 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73270bf6-9eac-4c54-afc8-bcb24eade858 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:05.329449 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-73270bf6-9eac-4c54-afc8-bcb24eade858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:05.345992 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73270bf6-9eac-4c54-afc8-bcb24eade858 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:05.351401 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c61fa-727d-40c7-8810-2f1b3d1822fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:05.353421 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c61fa-727d-40c7-8810-2f1b3d1822fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:05.353720 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c61fa-727d-40c7-8810-2f1b3d1822fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:05.354049 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c61fa-727d-40c7-8810-2f1b3d1822fd admin admin] PATH_INFO: `/v3/projects/1a0a7a2c316c418d853e83caa358832e` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:05.355245 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824c61fa-727d-40c7-8810-2f1b3d1822fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a0a7a2c316c418d853e83caa358832e)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:05.357190 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824c61fa-727d-40c7-8810-2f1b3d1822fd admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:05.474419 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-824c61fa-727d-40c7-8810-2f1b3d1822fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a0a7a2c316c418d853e83caa358832e', 'request_id': 'req-824c61fa-727d-40c7-8810-2f1b3d1822fd'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:05.481549 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-824c61fa-727d-40c7-8810-2f1b3d1822fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a0a7a2c316c418d853e83caa358832e', 'request_id': 'req-824c61fa-727d-40c7-8810-2f1b3d1822fd'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:05.487222 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-824c61fa-727d-40c7-8810-2f1b3d1822fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a0a7a2c316c418d853e83caa358832e', 'request_id': 'req-824c61fa-727d-40c7-8810-2f1b3d1822fd'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:05.594116 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-824c61fa-727d-40c7-8810-2f1b3d1822fd admin admin] The token cache is being invalidate because project 1a0a7a2c316c418d853e83caa358832e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:40:05.594405 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-824c61fa-727d-40c7-8810-2f1b3d1822fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:05.595048 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-824c61fa-727d-40c7-8810-2f1b3d1822fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:05.596655 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1418/2807] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:40:05 2022] DELETE /identity/v3/projects/1a0a7a2c316c418d853e83caa358832e => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:40:05.602660 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc977eb-4305-4624-8639-5cc46134a61c service neutron] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:05.612703 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc977eb-4305-4624-8639-5cc46134a61c service neutron] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:05.643353 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:40:05.677139 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f23b9b-6d1f-4793-8865-effd5babf149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:05.678671 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f23b9b-6d1f-4793-8865-effd5babf149 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:05.678966 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f23b9b-6d1f-4793-8865-effd5babf149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:05.679246 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f23b9b-6d1f-4793-8865-effd5babf149 admin admin] PATH_INFO: `/v3/users/6c689d3168814997a2104c9e7ecffa54` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:05.680563 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f23b9b-6d1f-4793-8865-effd5babf149 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c689d3168814997a2104c9e7ecffa54)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:05.682483 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f23b9b-6d1f-4793-8865-effd5babf149 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:05.906683 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-b2f23b9b-6d1f-4793-8865-effd5babf149 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c689d3168814997a2104c9e7ecffa54'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:05.919554 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-b2f23b9b-6d1f-4793-8865-effd5babf149 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c689d3168814997a2104c9e7ecffa54'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:05.920252 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-b2f23b9b-6d1f-4793-8865-effd5babf149 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c689d3168814997a2104c9e7ecffa54'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:05.927131 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-b2f23b9b-6d1f-4793-8865-effd5babf149 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c689d3168814997a2104c9e7ecffa54'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:05.928368 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-b2f23b9b-6d1f-4793-8865-effd5babf149 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c689d3168814997a2104c9e7ecffa54'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:06.005578 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1390/2808] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:40:05 2022] DELETE /identity/v3/users/6c689d3168814997a2104c9e7ecffa54 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:40:06.412962 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c61fa-727d-40c7-8810-2f1b3d1822fd admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:06.414417 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-824c61fa-727d-40c7-8810-2f1b3d1822fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:06.428626 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c61fa-727d-40c7-8810-2f1b3d1822fd admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:06.439946 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:40:06.488896 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834dcf69-7b9a-4445-a7ea-69d1599bfc18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:06.489795 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834dcf69-7b9a-4445-a7ea-69d1599bfc18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:06.490124 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834dcf69-7b9a-4445-a7ea-69d1599bfc18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:06.490353 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834dcf69-7b9a-4445-a7ea-69d1599bfc18 admin admin] PATH_INFO: `/v3/projects/8f5c39d5bdc948258e7f62dedb242138` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:06.491919 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834dcf69-7b9a-4445-a7ea-69d1599bfc18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f5c39d5bdc948258e7f62dedb242138)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:06.493845 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834dcf69-7b9a-4445-a7ea-69d1599bfc18 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:06.606159 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-834dcf69-7b9a-4445-a7ea-69d1599bfc18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f5c39d5bdc948258e7f62dedb242138', 'request_id': 'req-834dcf69-7b9a-4445-a7ea-69d1599bfc18'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:06.612380 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-834dcf69-7b9a-4445-a7ea-69d1599bfc18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f5c39d5bdc948258e7f62dedb242138', 'request_id': 'req-834dcf69-7b9a-4445-a7ea-69d1599bfc18'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:06.618045 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-834dcf69-7b9a-4445-a7ea-69d1599bfc18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f5c39d5bdc948258e7f62dedb242138', 'request_id': 'req-834dcf69-7b9a-4445-a7ea-69d1599bfc18'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:06.702525 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-834dcf69-7b9a-4445-a7ea-69d1599bfc18 admin admin] The token cache is being invalidate because project 8f5c39d5bdc948258e7f62dedb242138 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:40:06.702726 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-834dcf69-7b9a-4445-a7ea-69d1599bfc18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:06.703369 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-834dcf69-7b9a-4445-a7ea-69d1599bfc18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:06.705106 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1419/2809] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:40:06 2022] DELETE /identity/v3/projects/8f5c39d5bdc948258e7f62dedb242138 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:40:06.711666 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad0133-94a2-4b4a-8697-0e82b67d2e19 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:06.718790 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad0133-94a2-4b4a-8697-0e82b67d2e19 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:06.718790 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad0133-94a2-4b4a-8697-0e82b67d2e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:06.731493 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-f7ad0133-94a2-4b4a-8697-0e82b67d2e19 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:40:06.732774 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-f7ad0133-94a2-4b4a-8697-0e82b67d2e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:06.794184 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1391/2810] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:40:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:40:06.802021 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11daa4cc-65d6-4877-8e02-ded5cba22d72 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:06.802296 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11daa4cc-65d6-4877-8e02-ded5cba22d72 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:06.802578 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11daa4cc-65d6-4877-8e02-ded5cba22d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:06.822227 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-11daa4cc-65d6-4877-8e02-ded5cba22d72 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:40:06.823437 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-11daa4cc-65d6-4877-8e02-ded5cba22d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:06.831407 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1420/2811] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:40:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:40:06.836697 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ad0133-94a2-4b4a-8697-0e82b67d2e19 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:06.846369 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ad0133-94a2-4b4a-8697-0e82b67d2e19 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:06.850978 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336ae7b3-fc1e-4902-b4af-429f6ad32b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:06.851700 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336ae7b3-fc1e-4902-b4af-429f6ad32b0e admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:06.851984 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336ae7b3-fc1e-4902-b4af-429f6ad32b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:06.852263 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336ae7b3-fc1e-4902-b4af-429f6ad32b0e admin admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:06.853023 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336ae7b3-fc1e-4902-b4af-429f6ad32b0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:40:06.853358 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336ae7b3-fc1e-4902-b4af-429f6ad32b0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:06.854933 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336ae7b3-fc1e-4902-b4af-429f6ad32b0e admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:06.867072 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1392/2812] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:40:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:40:06.871312 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11daa4cc-65d6-4877-8e02-ded5cba22d72 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:06.880525 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11daa4cc-65d6-4877-8e02-ded5cba22d72 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:06.886207 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2279064a-e953-4df3-9afa-73507a1fcaed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:06.886874 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2279064a-e953-4df3-9afa-73507a1fcaed admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:06.887166 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2279064a-e953-4df3-9afa-73507a1fcaed admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:06.887444 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2279064a-e953-4df3-9afa-73507a1fcaed admin admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:06.888150 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2279064a-e953-4df3-9afa-73507a1fcaed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:06.890083 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2279064a-e953-4df3-9afa-73507a1fcaed admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:07.045653 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1421/2813] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:40:06 2022] POST /identity/v3/projects => generated 407 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:40:07.049482 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336ae7b3-fc1e-4902-b4af-429f6ad32b0e admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:07.060306 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336ae7b3-fc1e-4902-b4af-429f6ad32b0e admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:07.095291 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:40:07.122774 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8c09d6-e550-444d-9a70-a67a12e8039b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:07.123441 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8c09d6-e550-444d-9a70-a67a12e8039b admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:07.123744 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8c09d6-e550-444d-9a70-a67a12e8039b admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:07.124002 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8c09d6-e550-444d-9a70-a67a12e8039b admin admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:07.124774 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8c09d6-e550-444d-9a70-a67a12e8039b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:07.126702 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8c09d6-e550-444d-9a70-a67a12e8039b admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:07.214568 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1393/2814] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:40:07 2022] POST /identity/v3/users => generated 304 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:40:07.219035 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2279064a-e953-4df3-9afa-73507a1fcaed admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:07.229995 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2279064a-e953-4df3-9afa-73507a1fcaed admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:07.235259 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15d15d5-ad70-4c11-80cd-5a2dea384477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:07.235929 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d15d5-ad70-4c11-80cd-5a2dea384477 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:07.236237 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d15d5-ad70-4c11-80cd-5a2dea384477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:07.236551 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d15d5-ad70-4c11-80cd-5a2dea384477 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:07.237282 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15d15d5-ad70-4c11-80cd-5a2dea384477 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:07.238889 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15d15d5-ad70-4c11-80cd-5a2dea384477 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:07.250755 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1422/2815] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:40:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:40:07.255120 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8c09d6-e550-444d-9a70-a67a12e8039b admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:07.266622 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8c09d6-e550-444d-9a70-a67a12e8039b admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:07.271415 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c0ea4c-ede5-4401-8424-d4a0ec5601c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:07.272084 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c0ea4c-ede5-4401-8424-d4a0ec5601c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:07.272596 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c0ea4c-ede5-4401-8424-d4a0ec5601c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:07.272887 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c0ea4c-ede5-4401-8424-d4a0ec5601c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:07.273624 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c0ea4c-ede5-4401-8424-d4a0ec5601c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:07.275216 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c0ea4c-ede5-4401-8424-d4a0ec5601c8 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:07.285705 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1394/2816] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:40:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:40:07.290523 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15d15d5-ad70-4c11-80cd-5a2dea384477 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:07.300601 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15d15d5-ad70-4c11-80cd-5a2dea384477 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:07.305731 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef20e0d-6a5c-4ade-972b-a6c5f8aad3b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:07.306609 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef20e0d-6a5c-4ade-972b-a6c5f8aad3b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:07.306895 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef20e0d-6a5c-4ade-972b-a6c5f8aad3b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:07.307174 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef20e0d-6a5c-4ade-972b-a6c5f8aad3b1 admin admin] PATH_INFO: `/v3/projects/783ed9923ae74529bc8baa4f1c3829f3/users/f134eff5c616415db1ab68df17d62eb9/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:07.322369 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef20e0d-6a5c-4ade-972b-a6c5f8aad3b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=783ed9923ae74529bc8baa4f1c3829f3, user_id=f134eff5c616415db1ab68df17d62eb9, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:07.325062 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef20e0d-6a5c-4ade-972b-a6c5f8aad3b1 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:07.411876 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1423/2817] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:40:07 2022] PUT /identity/v3/projects/783ed9923ae74529bc8baa4f1c3829f3/users/f134eff5c616415db1ab68df17d62eb9/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:40:07.422523 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a79f09e-9faa-4395-8819-cf631a04cee5 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:07.422818 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a79f09e-9faa-4395-8819-cf631a04cee5 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:07.423099 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a79f09e-9faa-4395-8819-cf631a04cee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:07.451766 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-5a79f09e-9faa-4395-8819-cf631a04cee5 None None] MFA Rules not processed for user `f134eff5c616415db1ab68df17d62eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:40:07.453046 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-5a79f09e-9faa-4395-8819-cf631a04cee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:07.517807 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1395/2818] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:40:07 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:40:07.522424 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef20e0d-6a5c-4ade-972b-a6c5f8aad3b1 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:07.564170 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:40:07.585073 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef20e0d-6a5c-4ade-972b-a6c5f8aad3b1 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:07.589775 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2a8896-915a-4598-b649-87adfb5cac8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:07.590529 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2a8896-915a-4598-b649-87adfb5cac8f admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:07.591025 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2a8896-915a-4598-b649-87adfb5cac8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:07.591346 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2a8896-915a-4598-b649-87adfb5cac8f admin admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:07.592287 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2a8896-915a-4598-b649-87adfb5cac8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:07.594745 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2a8896-915a-4598-b649-87adfb5cac8f admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:07.713286 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1424/2819] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:40:07 2022] POST /identity/v3/projects => generated 407 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:40:07.717198 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a79f09e-9faa-4395-8819-cf631a04cee5 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:07.728970 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a79f09e-9faa-4395-8819-cf631a04cee5 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:07.784468 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95692841-5cd8-413b-83b0-4c2fd84b5b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:07.785139 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95692841-5cd8-413b-83b0-4c2fd84b5b90 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:07.785422 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95692841-5cd8-413b-83b0-4c2fd84b5b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:07.785696 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95692841-5cd8-413b-83b0-4c2fd84b5b90 admin admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:07.786387 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95692841-5cd8-413b-83b0-4c2fd84b5b90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:07.788826 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95692841-5cd8-413b-83b0-4c2fd84b5b90 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:07.877516 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1396/2820] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:40:07 2022] POST /identity/v3/users => generated 310 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:40:07.881864 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2a8896-915a-4598-b649-87adfb5cac8f admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:07.892853 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2a8896-915a-4598-b649-87adfb5cac8f admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:07.901933 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82561c0f-8cc9-4ab5-9782-246dd46c2998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:07.902601 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82561c0f-8cc9-4ab5-9782-246dd46c2998 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:07.902882 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82561c0f-8cc9-4ab5-9782-246dd46c2998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:07.903157 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82561c0f-8cc9-4ab5-9782-246dd46c2998 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:07.903885 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82561c0f-8cc9-4ab5-9782-246dd46c2998 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:07.905500 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82561c0f-8cc9-4ab5-9782-246dd46c2998 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:07.917058 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1425/2821] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:40:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:40:07.921415 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95692841-5cd8-413b-83b0-4c2fd84b5b90 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:07.932638 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95692841-5cd8-413b-83b0-4c2fd84b5b90 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:07.937486 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5084bbea-e4c5-409f-bb22-d704bda696f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:07.938380 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5084bbea-e4c5-409f-bb22-d704bda696f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:07.938665 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5084bbea-e4c5-409f-bb22-d704bda696f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:07.939085 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5084bbea-e4c5-409f-bb22-d704bda696f6 admin admin] PATH_INFO: `/v3/projects/798c4e608c294c71b2992e53dba9c4cf/users/a6c842514a154f7ca969aad2b0bad178/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:07.954902 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5084bbea-e4c5-409f-bb22-d704bda696f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=798c4e608c294c71b2992e53dba9c4cf, user_id=a6c842514a154f7ca969aad2b0bad178, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:07.957606 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5084bbea-e4c5-409f-bb22-d704bda696f6 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:08.029402 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1397/2822] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:40:07 2022] PUT /identity/v3/projects/798c4e608c294c71b2992e53dba9c4cf/users/a6c842514a154f7ca969aad2b0bad178/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:40:08.033454 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82561c0f-8cc9-4ab5-9782-246dd46c2998 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:08.044266 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82561c0f-8cc9-4ab5-9782-246dd46c2998 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:08.098872 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67339cc6-d913-474c-b184-65af675cc7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:08.099540 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67339cc6-d913-474c-b184-65af675cc7ff admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:08.100490 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67339cc6-d913-474c-b184-65af675cc7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:08.100769 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67339cc6-d913-474c-b184-65af675cc7ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:08.101495 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67339cc6-d913-474c-b184-65af675cc7ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:08.103122 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67339cc6-d913-474c-b184-65af675cc7ff admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:08.112742 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1426/2823] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:40:08 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:40:08.117019 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5084bbea-e4c5-409f-bb22-d704bda696f6 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:08.127635 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5084bbea-e4c5-409f-bb22-d704bda696f6 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:08.132267 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404961d9-5915-44de-a2ff-8f6edf11fd8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:08.133179 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404961d9-5915-44de-a2ff-8f6edf11fd8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:08.133462 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404961d9-5915-44de-a2ff-8f6edf11fd8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:08.133993 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404961d9-5915-44de-a2ff-8f6edf11fd8f admin admin] PATH_INFO: `/v3/projects/798c4e608c294c71b2992e53dba9c4cf/users/a6c842514a154f7ca969aad2b0bad178/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:08.135942 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404961d9-5915-44de-a2ff-8f6edf11fd8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=798c4e608c294c71b2992e53dba9c4cf, user_id=a6c842514a154f7ca969aad2b0bad178, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:08.138623 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404961d9-5915-44de-a2ff-8f6edf11fd8f admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:08.140854 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:40:08.150059 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1398/2824] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:40:08 2022] PUT /identity/v3/projects/798c4e608c294c71b2992e53dba9c4cf/users/a6c842514a154f7ca969aad2b0bad178/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:40:08.165113 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6133505c-cbc9-4a5a-bf3e-aa33e7cacc52 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:08.165410 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6133505c-cbc9-4a5a-bf3e-aa33e7cacc52 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:08.165752 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6133505c-cbc9-4a5a-bf3e-aa33e7cacc52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:08.192174 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-6133505c-cbc9-4a5a-bf3e-aa33e7cacc52 None None] MFA Rules not processed for user `a6c842514a154f7ca969aad2b0bad178`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:40:08.193425 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-6133505c-cbc9-4a5a-bf3e-aa33e7cacc52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:08.261622 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1427/2825] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:40:08 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:40:08.273384 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404961d9-5915-44de-a2ff-8f6edf11fd8f admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:08.275111 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-404961d9-5915-44de-a2ff-8f6edf11fd8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:08.332481 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404961d9-5915-44de-a2ff-8f6edf11fd8f admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:08.336769 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d83b84-5452-42cf-9ca7-9977fed0d414 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:08.337412 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d83b84-5452-42cf-9ca7-9977fed0d414 service cinder] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:08.337700 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d83b84-5452-42cf-9ca7-9977fed0d414 service cinder] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:08.337982 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d83b84-5452-42cf-9ca7-9977fed0d414 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:08.347005 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d83b84-5452-42cf-9ca7-9977fed0d414 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:08.348448 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d83b84-5452-42cf-9ca7-9977fed0d414 service cinder] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:08.358145 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1399/2826] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:40:08 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:40:08.687909 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6133505c-cbc9-4a5a-bf3e-aa33e7cacc52 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:08.698627 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6133505c-cbc9-4a5a-bf3e-aa33e7cacc52 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:08.702298 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdce6e5-2f08-44d0-aacc-e490b39a7812 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:08.702941 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdce6e5-2f08-44d0-aacc-e490b39a7812 service cinder] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:08.703226 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdce6e5-2f08-44d0-aacc-e490b39a7812 service cinder] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:08.703503 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdce6e5-2f08-44d0-aacc-e490b39a7812 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:08.735850 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:40:08.760920 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdce6e5-2f08-44d0-aacc-e490b39a7812 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:08.762353 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdce6e5-2f08-44d0-aacc-e490b39a7812 service cinder] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:08.771277 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1428/2827] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:40:08 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:40:14.113411 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d83b84-5452-42cf-9ca7-9977fed0d414 service cinder] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:14.114831 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-90d83b84-5452-42cf-9ca7-9977fed0d414 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:14.207810 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d83b84-5452-42cf-9ca7-9977fed0d414 service cinder] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:14.214189 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dd51af-eedb-4244-bf65-b8b255fb40dc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938187967efe49be9c5389d1feef81ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:14.214827 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dd51af-eedb-4244-bf65-b8b255fb40dc service placement] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:14.215117 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dd51af-eedb-4244-bf65-b8b255fb40dc service placement] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:14.215397 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dd51af-eedb-4244-bf65-b8b255fb40dc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:14.217386 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-84dd51af-eedb-4244-bf65-b8b255fb40dc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:14.230334 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dd51af-eedb-4244-bf65-b8b255fb40dc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:14.231909 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dd51af-eedb-4244-bf65-b8b255fb40dc service placement] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:14.242259 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1400/2828] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:40:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:40:18.810975 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=12 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:40:19.033644 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdce6e5-2f08-44d0-aacc-e490b39a7812 service cinder] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:19.105913 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdce6e5-2f08-44d0-aacc-e490b39a7812 service cinder] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:19.110503 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaf1f20-45fb-4d2f-8f34-fe75a8e47db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:19.111473 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf1f20-45fb-4d2f-8f34-fe75a8e47db4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:19.111737 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf1f20-45fb-4d2f-8f34-fe75a8e47db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:19.112011 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf1f20-45fb-4d2f-8f34-fe75a8e47db4 admin admin] PATH_INFO: `/v3/users/f134eff5c616415db1ab68df17d62eb9` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:19.113273 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaf1f20-45fb-4d2f-8f34-fe75a8e47db4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f134eff5c616415db1ab68df17d62eb9)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:19.115211 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaf1f20-45fb-4d2f-8f34-fe75a8e47db4 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:19.344771 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-9aaf1f20-45fb-4d2f-8f34-fe75a8e47db4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f134eff5c616415db1ab68df17d62eb9'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:19.358195 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-9aaf1f20-45fb-4d2f-8f34-fe75a8e47db4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f134eff5c616415db1ab68df17d62eb9'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:19.434405 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-9aaf1f20-45fb-4d2f-8f34-fe75a8e47db4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f134eff5c616415db1ab68df17d62eb9'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:19.435020 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-9aaf1f20-45fb-4d2f-8f34-fe75a8e47db4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f134eff5c616415db1ab68df17d62eb9'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:19.442575 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-9aaf1f20-45fb-4d2f-8f34-fe75a8e47db4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f134eff5c616415db1ab68df17d62eb9'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:19.448630 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1429/2829] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:40:19 2022] DELETE /identity/v3/users/f134eff5c616415db1ab68df17d62eb9 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:40:19.454200 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b743420-773f-417a-91fa-51c045daa11e None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:19.454753 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b743420-773f-417a-91fa-51c045daa11e None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:19.454862 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b743420-773f-417a-91fa-51c045daa11e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:19.474191 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-1b743420-773f-417a-91fa-51c045daa11e None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:40:19.476025 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-1b743420-773f-417a-91fa-51c045daa11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:19.539115 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1401/2830] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:40:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:40:19.549145 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaf1f20-45fb-4d2f-8f34-fe75a8e47db4 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:19.550755 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-9aaf1f20-45fb-4d2f-8f34-fe75a8e47db4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:19.613841 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaf1f20-45fb-4d2f-8f34-fe75a8e47db4 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:19.617897 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c48dc5c-d47a-499f-b3ca-9933c8b0e2f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:19.618569 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c48dc5c-d47a-499f-b3ca-9933c8b0e2f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:19.618855 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c48dc5c-d47a-499f-b3ca-9933c8b0e2f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:19.619181 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c48dc5c-d47a-499f-b3ca-9933c8b0e2f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:19.629817 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c48dc5c-d47a-499f-b3ca-9933c8b0e2f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:19.631955 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c48dc5c-d47a-499f-b3ca-9933c8b0e2f1 service neutron] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:19.642993 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1430/2831] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:40:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:40:20.084290 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b743420-773f-417a-91fa-51c045daa11e None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:20.086443 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-1b743420-773f-417a-91fa-51c045daa11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:20.116527 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b743420-773f-417a-91fa-51c045daa11e None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:20.122320 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc9ea28-6011-4bea-a2dc-b605e1c0fd9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:20.123140 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9ea28-6011-4bea-a2dc-b605e1c0fd9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:20.123497 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9ea28-6011-4bea-a2dc-b605e1c0fd9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:20.123782 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9ea28-6011-4bea-a2dc-b605e1c0fd9a admin admin] PATH_INFO: `/v3/projects/783ed9923ae74529bc8baa4f1c3829f3` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:20.125026 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc9ea28-6011-4bea-a2dc-b605e1c0fd9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=783ed9923ae74529bc8baa4f1c3829f3)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:20.127065 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc9ea28-6011-4bea-a2dc-b605e1c0fd9a admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:20.128819 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:40:20.236408 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-ccc9ea28-6011-4bea-a2dc-b605e1c0fd9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '783ed9923ae74529bc8baa4f1c3829f3', 'request_id': 'req-ccc9ea28-6011-4bea-a2dc-b605e1c0fd9a'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:20.291204 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-ccc9ea28-6011-4bea-a2dc-b605e1c0fd9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '783ed9923ae74529bc8baa4f1c3829f3', 'request_id': 'req-ccc9ea28-6011-4bea-a2dc-b605e1c0fd9a'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:20.298332 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-ccc9ea28-6011-4bea-a2dc-b605e1c0fd9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '783ed9923ae74529bc8baa4f1c3829f3', 'request_id': 'req-ccc9ea28-6011-4bea-a2dc-b605e1c0fd9a'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:20.310666 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-ccc9ea28-6011-4bea-a2dc-b605e1c0fd9a admin admin] The token cache is being invalidate because project 783ed9923ae74529bc8baa4f1c3829f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:40:20.310959 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-ccc9ea28-6011-4bea-a2dc-b605e1c0fd9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:20.311635 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-ccc9ea28-6011-4bea-a2dc-b605e1c0fd9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:20.313251 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1402/2832] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:40:20 2022] DELETE /identity/v3/projects/783ed9923ae74529bc8baa4f1c3829f3 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:40:20.317379 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c48dc5c-d47a-499f-b3ca-9933c8b0e2f1 service neutron] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:20.328633 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c48dc5c-d47a-499f-b3ca-9933c8b0e2f1 service neutron] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:20.347492 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:40:20.387041 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2584be1f-e131-470f-93a7-dbbb737553fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:20.387841 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2584be1f-e131-470f-93a7-dbbb737553fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:20.388129 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2584be1f-e131-470f-93a7-dbbb737553fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:20.388436 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2584be1f-e131-470f-93a7-dbbb737553fa admin admin] PATH_INFO: `/v3/users/a6c842514a154f7ca969aad2b0bad178` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:20.389650 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2584be1f-e131-470f-93a7-dbbb737553fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6c842514a154f7ca969aad2b0bad178)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:20.391617 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2584be1f-e131-470f-93a7-dbbb737553fa admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:20.565122 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-2584be1f-e131-470f-93a7-dbbb737553fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6c842514a154f7ca969aad2b0bad178'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:20.578052 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-2584be1f-e131-470f-93a7-dbbb737553fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6c842514a154f7ca969aad2b0bad178'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:20.665963 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-2584be1f-e131-470f-93a7-dbbb737553fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6c842514a154f7ca969aad2b0bad178'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:20.666622 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-2584be1f-e131-470f-93a7-dbbb737553fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6c842514a154f7ca969aad2b0bad178'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:20.675077 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-2584be1f-e131-470f-93a7-dbbb737553fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6c842514a154f7ca969aad2b0bad178'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:20.686564 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1431/2833] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:40:20 2022] DELETE /identity/v3/users/a6c842514a154f7ca969aad2b0bad178 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:40:21.129949 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc9ea28-6011-4bea-a2dc-b605e1c0fd9a admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:21.132526 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-ccc9ea28-6011-4bea-a2dc-b605e1c0fd9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:21.146437 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc9ea28-6011-4bea-a2dc-b605e1c0fd9a admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:21.213150 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cdeafe-9b4b-4382-b658-2ccd74003c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:21.213963 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cdeafe-9b4b-4382-b658-2ccd74003c8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:21.214244 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cdeafe-9b4b-4382-b658-2ccd74003c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:21.214520 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cdeafe-9b4b-4382-b658-2ccd74003c8b admin admin] PATH_INFO: `/v3/projects/798c4e608c294c71b2992e53dba9c4cf` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:21.215798 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cdeafe-9b4b-4382-b658-2ccd74003c8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=798c4e608c294c71b2992e53dba9c4cf)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:21.217762 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cdeafe-9b4b-4382-b658-2ccd74003c8b admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:21.355078 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-01cdeafe-9b4b-4382-b658-2ccd74003c8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '798c4e608c294c71b2992e53dba9c4cf', 'request_id': 'req-01cdeafe-9b4b-4382-b658-2ccd74003c8b'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:21.425756 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-01cdeafe-9b4b-4382-b658-2ccd74003c8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '798c4e608c294c71b2992e53dba9c4cf', 'request_id': 'req-01cdeafe-9b4b-4382-b658-2ccd74003c8b'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:21.433896 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-01cdeafe-9b4b-4382-b658-2ccd74003c8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '798c4e608c294c71b2992e53dba9c4cf', 'request_id': 'req-01cdeafe-9b4b-4382-b658-2ccd74003c8b'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:21.447834 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-01cdeafe-9b4b-4382-b658-2ccd74003c8b admin admin] The token cache is being invalidate because project 798c4e608c294c71b2992e53dba9c4cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:40:21.448237 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-01cdeafe-9b4b-4382-b658-2ccd74003c8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:21.448893 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-01cdeafe-9b4b-4382-b658-2ccd74003c8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:21.450915 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1403/2834] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:40:21 2022] DELETE /identity/v3/projects/798c4e608c294c71b2992e53dba9c4cf => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:40:21.456212 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00e789-7acf-4282-966a-a1770039591e None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:21.456916 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00e789-7acf-4282-966a-a1770039591e None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:21.456916 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00e789-7acf-4282-966a-a1770039591e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:21.476871 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-ab00e789-7acf-4282-966a-a1770039591e None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:40:21.478030 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-ab00e789-7acf-4282-966a-a1770039591e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:21.539032 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1432/2835] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:40:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:40:21.547403 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78060d5-e0c3-4b39-b144-da8e2313186f None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:21.547694 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78060d5-e0c3-4b39-b144-da8e2313186f None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:21.547977 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78060d5-e0c3-4b39-b144-da8e2313186f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:21.566422 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-d78060d5-e0c3-4b39-b144-da8e2313186f None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:40:21.567626 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-d78060d5-e0c3-4b39-b144-da8e2313186f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:21.576444 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1404/2836] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:40:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:40:21.581080 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab00e789-7acf-4282-966a-a1770039591e None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:21.593786 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab00e789-7acf-4282-966a-a1770039591e None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:21.599315 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57f46b6-f2a7-449a-b3d8-d68315e27cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:21.600016 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57f46b6-f2a7-449a-b3d8-d68315e27cbc admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:21.600672 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57f46b6-f2a7-449a-b3d8-d68315e27cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:21.600841 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57f46b6-f2a7-449a-b3d8-d68315e27cbc admin admin] PATH_INFO: `/v3/domains` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:21.601543 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57f46b6-f2a7-449a-b3d8-d68315e27cbc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:40:21.601853 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57f46b6-f2a7-449a-b3d8-d68315e27cbc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:21.603488 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57f46b6-f2a7-449a-b3d8-d68315e27cbc admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:21.616764 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1433/2837] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:40:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:40:21.621632 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78060d5-e0c3-4b39-b144-da8e2313186f None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:21.633447 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78060d5-e0c3-4b39-b144-da8e2313186f None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:21.638881 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aca9578-d8ee-44a2-a020-36c2531ccc4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:21.639551 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca9578-d8ee-44a2-a020-36c2531ccc4a admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:21.639837 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca9578-d8ee-44a2-a020-36c2531ccc4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:21.640113 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca9578-d8ee-44a2-a020-36c2531ccc4a admin admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:21.640933 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aca9578-d8ee-44a2-a020-36c2531ccc4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:21.642853 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aca9578-d8ee-44a2-a020-36c2531ccc4a admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:21.654113 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:40:21.739547 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1405/2838] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:40:21 2022] POST /identity/v3/projects => generated 377 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:40:21.744327 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57f46b6-f2a7-449a-b3d8-d68315e27cbc admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:21.754630 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57f46b6-f2a7-449a-b3d8-d68315e27cbc admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:21.786913 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:40:21.830698 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ae88f6-7c4f-4462-87e0-cf3f00449769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:21.831393 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ae88f6-7c4f-4462-87e0-cf3f00449769 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:21.831685 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ae88f6-7c4f-4462-87e0-cf3f00449769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:21.831968 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ae88f6-7c4f-4462-87e0-cf3f00449769 admin admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:21.832761 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ae88f6-7c4f-4462-87e0-cf3f00449769 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:21.834708 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ae88f6-7c4f-4462-87e0-cf3f00449769 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:21.935115 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1434/2839] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:40:21 2022] POST /identity/v3/users => generated 289 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:40:21.939259 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aca9578-d8ee-44a2-a020-36c2531ccc4a admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:21.949969 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aca9578-d8ee-44a2-a020-36c2531ccc4a admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:21.954730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e520721-5381-4506-8c48-4f42f2de0bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:21.955429 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e520721-5381-4506-8c48-4f42f2de0bec admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:21.965580 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e520721-5381-4506-8c48-4f42f2de0bec admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:21.965580 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e520721-5381-4506-8c48-4f42f2de0bec admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:21.965580 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e520721-5381-4506-8c48-4f42f2de0bec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:21.965580 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e520721-5381-4506-8c48-4f42f2de0bec admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:21.979662 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1406/2840] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:40:21 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:40:21.983612 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ae88f6-7c4f-4462-87e0-cf3f00449769 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:21.995332 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ae88f6-7c4f-4462-87e0-cf3f00449769 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:22.000736 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2326c2b7-797b-47a4-8118-0e94990a5d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:22.001438 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2326c2b7-797b-47a4-8118-0e94990a5d39 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:22.001724 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2326c2b7-797b-47a4-8118-0e94990a5d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:22.002003 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2326c2b7-797b-47a4-8118-0e94990a5d39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:22.002833 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2326c2b7-797b-47a4-8118-0e94990a5d39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:22.004527 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2326c2b7-797b-47a4-8118-0e94990a5d39 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:22.016986 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1435/2841] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:40:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:40:22.023678 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e520721-5381-4506-8c48-4f42f2de0bec admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:22.033244 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e520721-5381-4506-8c48-4f42f2de0bec admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:22.038809 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3277e4-5999-4ea0-8afa-28627caa2166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:22.039736 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3277e4-5999-4ea0-8afa-28627caa2166 admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:22.040020 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3277e4-5999-4ea0-8afa-28627caa2166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:22.040299 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3277e4-5999-4ea0-8afa-28627caa2166 admin admin] PATH_INFO: `/v3/projects/371d66914ecf42bf83ba3ddc714ff34a/users/0104626800e845b697ac5fdd38ae3597/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:22.057775 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3277e4-5999-4ea0-8afa-28627caa2166 admin admin] RBAC: Authorizing `identity:create_grant(project_id=371d66914ecf42bf83ba3ddc714ff34a, user_id=0104626800e845b697ac5fdd38ae3597, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:22.060596 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3277e4-5999-4ea0-8afa-28627caa2166 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:22.152800 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1407/2842] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:40:22 2022] PUT /identity/v3/projects/371d66914ecf42bf83ba3ddc714ff34a/users/0104626800e845b697ac5fdd38ae3597/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:40:22.165059 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00143db9-0be3-463e-bc57-68bbb01c9414 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:22.165482 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00143db9-0be3-463e-bc57-68bbb01c9414 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:22.165801 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00143db9-0be3-463e-bc57-68bbb01c9414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:22.194257 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-00143db9-0be3-463e-bc57-68bbb01c9414 None None] MFA Rules not processed for user `0104626800e845b697ac5fdd38ae3597`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:40:22.195571 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-00143db9-0be3-463e-bc57-68bbb01c9414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:22.259304 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1436/2843] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:40:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:40:22.263907 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3277e4-5999-4ea0-8afa-28627caa2166 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:22.326185 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3277e4-5999-4ea0-8afa-28627caa2166 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:22.343539 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a910f017-c980-465b-ac04-efbf20761e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:22.344246 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a910f017-c980-465b-ac04-efbf20761e8a admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:22.344572 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a910f017-c980-465b-ac04-efbf20761e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:22.344854 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a910f017-c980-465b-ac04-efbf20761e8a admin admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:22.345581 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a910f017-c980-465b-ac04-efbf20761e8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:22.347504 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a910f017-c980-465b-ac04-efbf20761e8a admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:22.456409 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1408/2844] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:40:22 2022] POST /identity/v3/projects => generated 375 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:40:22.461663 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00143db9-0be3-463e-bc57-68bbb01c9414 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:22.473614 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00143db9-0be3-463e-bc57-68bbb01c9414 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:22.535092 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac53fb5-acc5-476b-b56f-dea141b277ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:22.535688 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac53fb5-acc5-476b-b56f-dea141b277ff admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:22.535974 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac53fb5-acc5-476b-b56f-dea141b277ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:22.536262 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac53fb5-acc5-476b-b56f-dea141b277ff admin admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:22.537086 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac53fb5-acc5-476b-b56f-dea141b277ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:22.539512 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac53fb5-acc5-476b-b56f-dea141b277ff admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:22.612315 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1437/2845] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:40:22 2022] POST /identity/v3/users => generated 288 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:40:22.617046 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a910f017-c980-465b-ac04-efbf20761e8a admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:22.628988 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a910f017-c980-465b-ac04-efbf20761e8a admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:22.634154 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99161d77-bc0d-4be1-923c-f692f83e3476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:22.634867 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99161d77-bc0d-4be1-923c-f692f83e3476 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:22.635161 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99161d77-bc0d-4be1-923c-f692f83e3476 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:22.635433 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99161d77-bc0d-4be1-923c-f692f83e3476 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:22.636178 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99161d77-bc0d-4be1-923c-f692f83e3476 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:22.637847 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99161d77-bc0d-4be1-923c-f692f83e3476 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:22.654694 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1409/2846] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:40:22 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:40:22.658788 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac53fb5-acc5-476b-b56f-dea141b277ff admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:22.670396 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac53fb5-acc5-476b-b56f-dea141b277ff admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:22.675006 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff36c70-dddd-44e6-aded-5b8f433a5ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:22.675677 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff36c70-dddd-44e6-aded-5b8f433a5ab8 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:22.675958 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff36c70-dddd-44e6-aded-5b8f433a5ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:22.676231 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff36c70-dddd-44e6-aded-5b8f433a5ab8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:22.676986 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff36c70-dddd-44e6-aded-5b8f433a5ab8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:22.679829 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff36c70-dddd-44e6-aded-5b8f433a5ab8 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:22.696114 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1438/2847] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:40:22 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:40:22.700775 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99161d77-bc0d-4be1-923c-f692f83e3476 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:22.712537 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99161d77-bc0d-4be1-923c-f692f83e3476 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:22.717528 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596beaff-cc5a-4902-9c99-6724c6928ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:22.718422 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596beaff-cc5a-4902-9c99-6724c6928ddf admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:22.718744 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596beaff-cc5a-4902-9c99-6724c6928ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:22.718963 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596beaff-cc5a-4902-9c99-6724c6928ddf admin admin] PATH_INFO: `/v3/projects/b41776b5c7934b8295acd9785b1add57/users/89d1cd90e47644bf85aa4ca42484edf9/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:22.735553 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596beaff-cc5a-4902-9c99-6724c6928ddf admin admin] RBAC: Authorizing `identity:create_grant(project_id=b41776b5c7934b8295acd9785b1add57, user_id=89d1cd90e47644bf85aa4ca42484edf9, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:22.738211 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596beaff-cc5a-4902-9c99-6724c6928ddf admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:22.872762 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1410/2848] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:40:22 2022] PUT /identity/v3/projects/b41776b5c7934b8295acd9785b1add57/users/89d1cd90e47644bf85aa4ca42484edf9/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:40:22.884033 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b6ae21-5968-416e-99b2-fa0a0c039835 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:22.884583 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b6ae21-5968-416e-99b2-fa0a0c039835 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:22.884970 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b6ae21-5968-416e-99b2-fa0a0c039835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:22.915449 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-a7b6ae21-5968-416e-99b2-fa0a0c039835 None None] MFA Rules not processed for user `89d1cd90e47644bf85aa4ca42484edf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:40:22.916734 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-a7b6ae21-5968-416e-99b2-fa0a0c039835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:22.921039 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:40:22.986936 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1439/2849] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:40:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:40:22.991730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596beaff-cc5a-4902-9c99-6724c6928ddf admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:23.061741 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596beaff-cc5a-4902-9c99-6724c6928ddf admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:23.066456 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d4a8b9-334b-4881-8f5f-0ab75c72fc23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:23.067113 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d4a8b9-334b-4881-8f5f-0ab75c72fc23 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:23.067395 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d4a8b9-334b-4881-8f5f-0ab75c72fc23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:23.067670 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d4a8b9-334b-4881-8f5f-0ab75c72fc23 admin admin] PATH_INFO: `/v3/projects` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:23.068357 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d4a8b9-334b-4881-8f5f-0ab75c72fc23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:23.070801 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d4a8b9-334b-4881-8f5f-0ab75c72fc23 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:23.080709 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:40:23.238892 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1411/2850] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:40:22 2022] POST /identity/v3/projects => generated 375 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:40:23.243808 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b6ae21-5968-416e-99b2-fa0a0c039835 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:23.255047 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b6ae21-5968-416e-99b2-fa0a0c039835 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:23.312644 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d61f527-5604-4239-8658-b83bf9025703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:23.313322 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d61f527-5604-4239-8658-b83bf9025703 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:23.313607 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d61f527-5604-4239-8658-b83bf9025703 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:23.313885 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d61f527-5604-4239-8658-b83bf9025703 admin admin] PATH_INFO: `/v3/users` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:23.314592 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d61f527-5604-4239-8658-b83bf9025703 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:23.316534 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d61f527-5604-4239-8658-b83bf9025703 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:23.404274 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1440/2851] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:40:23 2022] POST /identity/v3/users => generated 294 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:40:23.408725 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d4a8b9-334b-4881-8f5f-0ab75c72fc23 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:23.419902 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d4a8b9-334b-4881-8f5f-0ab75c72fc23 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:23.424911 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8cd395-e076-4be4-96cf-ea843d1420ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:23.425579 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8cd395-e076-4be4-96cf-ea843d1420ae admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:23.425875 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8cd395-e076-4be4-96cf-ea843d1420ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:23.426153 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8cd395-e076-4be4-96cf-ea843d1420ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:23.426878 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8cd395-e076-4be4-96cf-ea843d1420ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:23.428686 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8cd395-e076-4be4-96cf-ea843d1420ae admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:23.439475 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1412/2852] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:40:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:40:23.444325 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d61f527-5604-4239-8658-b83bf9025703 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:23.455192 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d61f527-5604-4239-8658-b83bf9025703 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:23.461281 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4b7c63-4490-4d44-a3fa-affdcd005c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:23.462283 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b7c63-4490-4d44-a3fa-affdcd005c77 admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:23.463599 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b7c63-4490-4d44-a3fa-affdcd005c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:23.463599 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b7c63-4490-4d44-a3fa-affdcd005c77 admin admin] PATH_INFO: `/v3/projects/0731e6692f7940b0adf8b2cf0fd84dd0/users/42fe39a4e9534b4e9f01c3ff634fde17/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:23.484626 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4b7c63-4490-4d44-a3fa-affdcd005c77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0731e6692f7940b0adf8b2cf0fd84dd0, user_id=42fe39a4e9534b4e9f01c3ff634fde17, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:23.487324 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4b7c63-4490-4d44-a3fa-affdcd005c77 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:23.569751 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1441/2853] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:40:23 2022] PUT /identity/v3/projects/0731e6692f7940b0adf8b2cf0fd84dd0/users/42fe39a4e9534b4e9f01c3ff634fde17/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:40:23.573933 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8cd395-e076-4be4-96cf-ea843d1420ae admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:23.585432 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8cd395-e076-4be4-96cf-ea843d1420ae admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:23.646904 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ab2e31-77d4-4d7d-96cd-d3b3965c0872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:23.648465 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ab2e31-77d4-4d7d-96cd-d3b3965c0872 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:23.648625 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ab2e31-77d4-4d7d-96cd-d3b3965c0872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:23.648903 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ab2e31-77d4-4d7d-96cd-d3b3965c0872 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:23.649711 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ab2e31-77d4-4d7d-96cd-d3b3965c0872 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:23.651356 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ab2e31-77d4-4d7d-96cd-d3b3965c0872 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:23.662625 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1413/2854] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:40:23 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:40:23.667112 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4b7c63-4490-4d44-a3fa-affdcd005c77 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:23.677814 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4b7c63-4490-4d44-a3fa-affdcd005c77 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:23.683626 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c8cfa8-4086-4d43-8034-76a3a740a7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:23.684561 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c8cfa8-4086-4d43-8034-76a3a740a7a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:23.684849 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c8cfa8-4086-4d43-8034-76a3a740a7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:23.685124 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c8cfa8-4086-4d43-8034-76a3a740a7a6 admin admin] PATH_INFO: `/v3/projects/0731e6692f7940b0adf8b2cf0fd84dd0/users/42fe39a4e9534b4e9f01c3ff634fde17/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:23.687315 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c8cfa8-4086-4d43-8034-76a3a740a7a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0731e6692f7940b0adf8b2cf0fd84dd0, user_id=42fe39a4e9534b4e9f01c3ff634fde17, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:23.690055 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c8cfa8-4086-4d43-8034-76a3a740a7a6 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:23.702312 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1442/2855] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:40:23 2022] PUT /identity/v3/projects/0731e6692f7940b0adf8b2cf0fd84dd0/users/42fe39a4e9534b4e9f01c3ff634fde17/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:40:23.712979 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29485644-e40a-4b24-9f99-54b69673a3ef None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:23.713275 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29485644-e40a-4b24-9f99-54b69673a3ef None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:23.713559 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29485644-e40a-4b24-9f99-54b69673a3ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:23.743038 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-29485644-e40a-4b24-9f99-54b69673a3ef None None] MFA Rules not processed for user `42fe39a4e9534b4e9f01c3ff634fde17`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:40:23.744254 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-29485644-e40a-4b24-9f99-54b69673a3ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:23.813986 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1414/2856] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:40:23 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:40:23.825975 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c8cfa8-4086-4d43-8034-76a3a740a7a6 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:23.827755 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-88c8cfa8-4086-4d43-8034-76a3a740a7a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:23.886620 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c8cfa8-4086-4d43-8034-76a3a740a7a6 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:23.890769 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8798e63-ed71-42d6-8a3f-f1805e022253 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:23.891471 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8798e63-ed71-42d6-8a3f-f1805e022253 service cinder] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:23.891761 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8798e63-ed71-42d6-8a3f-f1805e022253 service cinder] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:23.892040 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8798e63-ed71-42d6-8a3f-f1805e022253 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:23.903702 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:40:23.960018 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8798e63-ed71-42d6-8a3f-f1805e022253 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:23.961673 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8798e63-ed71-42d6-8a3f-f1805e022253 service cinder] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:23.970885 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1443/2857] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:40:23 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:40:26.256493 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29485644-e40a-4b24-9f99-54b69673a3ef None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:26.257984 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-29485644-e40a-4b24-9f99-54b69673a3ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:26.270408 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29485644-e40a-4b24-9f99-54b69673a3ef None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:26.274903 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24c8279-7ff3-4934-8515-93f910de4573 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:26.275545 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c8279-7ff3-4934-8515-93f910de4573 service cinder] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:26.275828 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c8279-7ff3-4934-8515-93f910de4573 service cinder] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:26.276106 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c8279-7ff3-4934-8515-93f910de4573 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:26.335277 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24c8279-7ff3-4934-8515-93f910de4573 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:26.336763 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24c8279-7ff3-4934-8515-93f910de4573 service cinder] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:26.341377 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:40:26.345969 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1415/2858] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:40:26 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:40:27.794654 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8798e63-ed71-42d6-8a3f-f1805e022253 service cinder] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:27.803828 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8798e63-ed71-42d6-8a3f-f1805e022253 service cinder] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:27.808593 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd0ca84-7262-4f8a-8374-6d5533307fb6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77c6e43555944902a22ba2fe17a8e99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:27.809669 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd0ca84-7262-4f8a-8374-6d5533307fb6 service cinder] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:27.810096 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd0ca84-7262-4f8a-8374-6d5533307fb6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:27.810378 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd0ca84-7262-4f8a-8374-6d5533307fb6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:27.823464 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd0ca84-7262-4f8a-8374-6d5533307fb6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:27.824939 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd0ca84-7262-4f8a-8374-6d5533307fb6 service cinder] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:27.835640 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1444/2859] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:40:27 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:40:32.390788 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24c8279-7ff3-4934-8515-93f910de4573 service cinder] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:32.452972 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24c8279-7ff3-4934-8515-93f910de4573 service cinder] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:32.457556 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74d15cc-87a9-49e4-a06c-4929ec88a229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:32.458433 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d15cc-87a9-49e4-a06c-4929ec88a229 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:32.458718 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d15cc-87a9-49e4-a06c-4929ec88a229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:32.458995 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d15cc-87a9-49e4-a06c-4929ec88a229 admin admin] PATH_INFO: `/v3/users/0104626800e845b697ac5fdd38ae3597` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:32.460185 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74d15cc-87a9-49e4-a06c-4929ec88a229 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0104626800e845b697ac5fdd38ae3597)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:32.462155 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74d15cc-87a9-49e4-a06c-4929ec88a229 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:32.714273 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-c74d15cc-87a9-49e4-a06c-4929ec88a229 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0104626800e845b697ac5fdd38ae3597'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:32.727391 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-c74d15cc-87a9-49e4-a06c-4929ec88a229 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0104626800e845b697ac5fdd38ae3597'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:32.730427 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-c74d15cc-87a9-49e4-a06c-4929ec88a229 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0104626800e845b697ac5fdd38ae3597'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:32.735222 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-c74d15cc-87a9-49e4-a06c-4929ec88a229 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0104626800e845b697ac5fdd38ae3597'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:32.735888 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-c74d15cc-87a9-49e4-a06c-4929ec88a229 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0104626800e845b697ac5fdd38ae3597'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:32.802817 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1416/2860] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:40:32 2022] DELETE /identity/v3/users/0104626800e845b697ac5fdd38ae3597 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:40:32.807473 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e7b476-ca49-4200-9ac4-0b1ae30d0e29 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:32.807971 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e7b476-ca49-4200-9ac4-0b1ae30d0e29 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:32.808200 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e7b476-ca49-4200-9ac4-0b1ae30d0e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:32.826647 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-d4e7b476-ca49-4200-9ac4-0b1ae30d0e29 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:40:32.827887 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-d4e7b476-ca49-4200-9ac4-0b1ae30d0e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:32.889443 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1445/2861] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:40:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:40:32.899766 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74d15cc-87a9-49e4-a06c-4929ec88a229 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:32.901209 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-c74d15cc-87a9-49e4-a06c-4929ec88a229 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:32.957140 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74d15cc-87a9-49e4-a06c-4929ec88a229 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:32.960647 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0491cba0-d752-4abf-b025-12d2ce98e229 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:32.961792 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0491cba0-d752-4abf-b025-12d2ce98e229 service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:32.962071 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0491cba0-d752-4abf-b025-12d2ce98e229 service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:32.962353 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0491cba0-d752-4abf-b025-12d2ce98e229 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:32.971507 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0491cba0-d752-4abf-b025-12d2ce98e229 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:32.972960 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0491cba0-d752-4abf-b025-12d2ce98e229 service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:32.983293 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1417/2862] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:40:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:40:33.401966 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e7b476-ca49-4200-9ac4-0b1ae30d0e29 None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:33.404600 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-d4e7b476-ca49-4200-9ac4-0b1ae30d0e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:33.418985 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e7b476-ca49-4200-9ac4-0b1ae30d0e29 None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:33.423842 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9046c56b-7b9f-4a83-9f04-681d3dbc096b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:33.424732 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046c56b-7b9f-4a83-9f04-681d3dbc096b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:33.425158 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046c56b-7b9f-4a83-9f04-681d3dbc096b admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:33.425313 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046c56b-7b9f-4a83-9f04-681d3dbc096b admin admin] PATH_INFO: `/v3/projects/371d66914ecf42bf83ba3ddc714ff34a` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:33.426478 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9046c56b-7b9f-4a83-9f04-681d3dbc096b admin admin] RBAC: Authorizing `identity:delete_project(project_id=371d66914ecf42bf83ba3ddc714ff34a)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:33.428445 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9046c56b-7b9f-4a83-9f04-681d3dbc096b admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:33.544076 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-9046c56b-7b9f-4a83-9f04-681d3dbc096b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '371d66914ecf42bf83ba3ddc714ff34a', 'request_id': 'req-9046c56b-7b9f-4a83-9f04-681d3dbc096b'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:33.550161 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-9046c56b-7b9f-4a83-9f04-681d3dbc096b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '371d66914ecf42bf83ba3ddc714ff34a', 'request_id': 'req-9046c56b-7b9f-4a83-9f04-681d3dbc096b'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:33.556189 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-9046c56b-7b9f-4a83-9f04-681d3dbc096b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '371d66914ecf42bf83ba3ddc714ff34a', 'request_id': 'req-9046c56b-7b9f-4a83-9f04-681d3dbc096b'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:33.630498 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-9046c56b-7b9f-4a83-9f04-681d3dbc096b admin admin] The token cache is being invalidate because project 371d66914ecf42bf83ba3ddc714ff34a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:40:33.630788 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-9046c56b-7b9f-4a83-9f04-681d3dbc096b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:33.631425 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-9046c56b-7b9f-4a83-9f04-681d3dbc096b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:33.633173 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1446/2863] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:40:33 2022] DELETE /identity/v3/projects/371d66914ecf42bf83ba3ddc714ff34a => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:40:33.638259 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0491cba0-d752-4abf-b025-12d2ce98e229 service neutron] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:33.648301 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0491cba0-d752-4abf-b025-12d2ce98e229 service neutron] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:33.670100 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:40:33.707993 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ec605-c734-423a-a364-90dc7ffc624f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:33.712605 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ec605-c734-423a-a364-90dc7ffc624f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:33.712605 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ec605-c734-423a-a364-90dc7ffc624f admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:33.712605 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ec605-c734-423a-a364-90dc7ffc624f admin admin] PATH_INFO: `/v3/users/89d1cd90e47644bf85aa4ca42484edf9` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:33.712605 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1ec605-c734-423a-a364-90dc7ffc624f admin admin] RBAC: Authorizing `identity:delete_user(user_id=89d1cd90e47644bf85aa4ca42484edf9)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:33.713371 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1ec605-c734-423a-a364-90dc7ffc624f admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:33.934380 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-0d1ec605-c734-423a-a364-90dc7ffc624f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89d1cd90e47644bf85aa4ca42484edf9'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:33.946872 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-0d1ec605-c734-423a-a364-90dc7ffc624f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89d1cd90e47644bf85aa4ca42484edf9'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:33.947558 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-0d1ec605-c734-423a-a364-90dc7ffc624f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89d1cd90e47644bf85aa4ca42484edf9'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:33.954404 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-0d1ec605-c734-423a-a364-90dc7ffc624f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89d1cd90e47644bf85aa4ca42484edf9'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:33.958830 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-0d1ec605-c734-423a-a364-90dc7ffc624f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89d1cd90e47644bf85aa4ca42484edf9'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:34.040783 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1418/2864] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:40:33 2022] DELETE /identity/v3/users/89d1cd90e47644bf85aa4ca42484edf9 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:40:34.467830 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9046c56b-7b9f-4a83-9f04-681d3dbc096b admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:34.469242 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-9046c56b-7b9f-4a83-9f04-681d3dbc096b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:34.483468 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9046c56b-7b9f-4a83-9f04-681d3dbc096b admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:34.502949 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:40:34.542073 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8f1c9c-58f3-4229-a81a-3cc25c16d728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:34.542938 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8f1c9c-58f3-4229-a81a-3cc25c16d728 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:34.543226 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8f1c9c-58f3-4229-a81a-3cc25c16d728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:34.543505 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8f1c9c-58f3-4229-a81a-3cc25c16d728 admin admin] PATH_INFO: `/v3/projects/b41776b5c7934b8295acd9785b1add57` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:34.544910 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8f1c9c-58f3-4229-a81a-3cc25c16d728 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b41776b5c7934b8295acd9785b1add57)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:34.547132 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8f1c9c-58f3-4229-a81a-3cc25c16d728 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:34.685962 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-ba8f1c9c-58f3-4229-a81a-3cc25c16d728 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b41776b5c7934b8295acd9785b1add57', 'request_id': 'req-ba8f1c9c-58f3-4229-a81a-3cc25c16d728'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:34.692602 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-ba8f1c9c-58f3-4229-a81a-3cc25c16d728 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b41776b5c7934b8295acd9785b1add57', 'request_id': 'req-ba8f1c9c-58f3-4229-a81a-3cc25c16d728'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:34.698291 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-ba8f1c9c-58f3-4229-a81a-3cc25c16d728 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b41776b5c7934b8295acd9785b1add57', 'request_id': 'req-ba8f1c9c-58f3-4229-a81a-3cc25c16d728'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:34.774237 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-ba8f1c9c-58f3-4229-a81a-3cc25c16d728 admin admin] The token cache is being invalidate because project b41776b5c7934b8295acd9785b1add57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:40:34.774237 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-ba8f1c9c-58f3-4229-a81a-3cc25c16d728 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:34.774781 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-ba8f1c9c-58f3-4229-a81a-3cc25c16d728 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:34.776336 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1447/2865] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:40:34 2022] DELETE /identity/v3/projects/b41776b5c7934b8295acd9785b1add57 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:40:34.780490 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ec605-c734-423a-a364-90dc7ffc624f admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:34.782024 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-0d1ec605-c734-423a-a364-90dc7ffc624f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:34.797254 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ec605-c734-423a-a364-90dc7ffc624f admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:34.856436 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34225cbb-25ae-44ba-98d5-e811abfd7ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:34.857368 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34225cbb-25ae-44ba-98d5-e811abfd7ff9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:34.857690 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34225cbb-25ae-44ba-98d5-e811abfd7ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:34.858161 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34225cbb-25ae-44ba-98d5-e811abfd7ff9 admin admin] PATH_INFO: `/v3/users/42fe39a4e9534b4e9f01c3ff634fde17` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:34.860016 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34225cbb-25ae-44ba-98d5-e811abfd7ff9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42fe39a4e9534b4e9f01c3ff634fde17)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:34.861980 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34225cbb-25ae-44ba-98d5-e811abfd7ff9 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:35.049380 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:40:35.051859 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-34225cbb-25ae-44ba-98d5-e811abfd7ff9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42fe39a4e9534b4e9f01c3ff634fde17'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:35.063390 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-34225cbb-25ae-44ba-98d5-e811abfd7ff9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42fe39a4e9534b4e9f01c3ff634fde17'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:35.064089 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-34225cbb-25ae-44ba-98d5-e811abfd7ff9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42fe39a4e9534b4e9f01c3ff634fde17'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:35.070787 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-34225cbb-25ae-44ba-98d5-e811abfd7ff9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42fe39a4e9534b4e9f01c3ff634fde17'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:35.071468 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-34225cbb-25ae-44ba-98d5-e811abfd7ff9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42fe39a4e9534b4e9f01c3ff634fde17'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:35.145550 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1419/2866] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:40:34 2022] DELETE /identity/v3/users/42fe39a4e9534b4e9f01c3ff634fde17 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:40:35.555660 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8f1c9c-58f3-4229-a81a-3cc25c16d728 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:35.559742 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-ba8f1c9c-58f3-4229-a81a-3cc25c16d728 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:35.569881 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8f1c9c-58f3-4229-a81a-3cc25c16d728 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:35.638749 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa548f39-9bf9-4ea4-8c08-af479ff54023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:35.639616 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa548f39-9bf9-4ea4-8c08-af479ff54023 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:35.639909 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa548f39-9bf9-4ea4-8c08-af479ff54023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:35.640187 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa548f39-9bf9-4ea4-8c08-af479ff54023 admin admin] PATH_INFO: `/v3/projects/0731e6692f7940b0adf8b2cf0fd84dd0` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:35.641493 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa548f39-9bf9-4ea4-8c08-af479ff54023 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0731e6692f7940b0adf8b2cf0fd84dd0)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:35.643382 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa548f39-9bf9-4ea4-8c08-af479ff54023 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:35.813769 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-fa548f39-9bf9-4ea4-8c08-af479ff54023 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0731e6692f7940b0adf8b2cf0fd84dd0', 'request_id': 'req-fa548f39-9bf9-4ea4-8c08-af479ff54023'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:35.820013 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-fa548f39-9bf9-4ea4-8c08-af479ff54023 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0731e6692f7940b0adf8b2cf0fd84dd0', 'request_id': 'req-fa548f39-9bf9-4ea4-8c08-af479ff54023'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:35.822148 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:40:35.827548 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-fa548f39-9bf9-4ea4-8c08-af479ff54023 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0731e6692f7940b0adf8b2cf0fd84dd0', 'request_id': 'req-fa548f39-9bf9-4ea4-8c08-af479ff54023'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:35.979380 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-fa548f39-9bf9-4ea4-8c08-af479ff54023 admin admin] The token cache is being invalidate because project 0731e6692f7940b0adf8b2cf0fd84dd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:40:35.979666 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-fa548f39-9bf9-4ea4-8c08-af479ff54023 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:35.980271 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-fa548f39-9bf9-4ea4-8c08-af479ff54023 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:40:35.981923 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1448/2867] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:40:35 2022] DELETE /identity/v3/projects/0731e6692f7940b0adf8b2cf0fd84dd0 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:40:35.986967 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c7004-8bad-4184-9835-adc3e9c9c8dd None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:35.987480 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c7004-8bad-4184-9835-adc3e9c9c8dd None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:35.987583 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c7004-8bad-4184-9835-adc3e9c9c8dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:36.006234 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-818c7004-8bad-4184-9835-adc3e9c9c8dd None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:40:36.011635 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-818c7004-8bad-4184-9835-adc3e9c9c8dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:36.068809 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1420/2868] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:40:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:40:36.130113 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6de06-971d-4291-9dff-5ade41bb406b None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:36.130396 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6de06-971d-4291-9dff-5ade41bb406b None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:36.130676 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6de06-971d-4291-9dff-5ade41bb406b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:36.150606 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-1cd6de06-971d-4291-9dff-5ade41bb406b None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:40:36.151800 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-1cd6de06-971d-4291-9dff-5ade41bb406b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:36.159962 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1449/2869] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:40:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:40:36.164730 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818c7004-8bad-4184-9835-adc3e9c9c8dd None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:36.174534 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818c7004-8bad-4184-9835-adc3e9c9c8dd None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:36.179037 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8fd138-7842-465e-8e50-00b7ef30befa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:36.180643 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8fd138-7842-465e-8e50-00b7ef30befa admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:36.180787 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8fd138-7842-465e-8e50-00b7ef30befa admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:36.181228 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8fd138-7842-465e-8e50-00b7ef30befa admin admin] PATH_INFO: `/v3/domains` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:36.181951 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8fd138-7842-465e-8e50-00b7ef30befa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:40:36.182279 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8fd138-7842-465e-8e50-00b7ef30befa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:36.184006 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8fd138-7842-465e-8e50-00b7ef30befa admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:36.195510 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1421/2870] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jun 21 18:40:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:40:36.199777 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd6de06-971d-4291-9dff-5ade41bb406b None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:36.209156 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd6de06-971d-4291-9dff-5ade41bb406b None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:36.214170 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95ab645-6c55-492e-a5de-abe2caad0c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:36.214832 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ab645-6c55-492e-a5de-abe2caad0c89 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:36.215120 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ab645-6c55-492e-a5de-abe2caad0c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:36.215400 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ab645-6c55-492e-a5de-abe2caad0c89 admin admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:36.216093 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95ab645-6c55-492e-a5de-abe2caad0c89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:36.218023 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95ab645-6c55-492e-a5de-abe2caad0c89 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:36.370587 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1450/2871] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:40:36 2022] POST /identity/v3/projects => generated 379 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:40:36.374806 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8fd138-7842-465e-8e50-00b7ef30befa admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:36.385182 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8fd138-7842-465e-8e50-00b7ef30befa admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:36.442512 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0636bf29-7337-4296-982e-fd7f1f36b74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:36.443105 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0636bf29-7337-4296-982e-fd7f1f36b74a admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:36.443395 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0636bf29-7337-4296-982e-fd7f1f36b74a admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:36.443705 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0636bf29-7337-4296-982e-fd7f1f36b74a admin admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:36.444419 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0636bf29-7337-4296-982e-fd7f1f36b74a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:36.446666 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0636bf29-7337-4296-982e-fd7f1f36b74a admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:36.574310 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1422/2872] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:40:36 2022] POST /identity/v3/users => generated 290 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:40:36.578893 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95ab645-6c55-492e-a5de-abe2caad0c89 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:36.590217 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95ab645-6c55-492e-a5de-abe2caad0c89 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:36.594793 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee489a8-2573-480d-88ab-96e59d689de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:36.595454 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee489a8-2573-480d-88ab-96e59d689de2 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:36.595739 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee489a8-2573-480d-88ab-96e59d689de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:36.596018 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee489a8-2573-480d-88ab-96e59d689de2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:36.596775 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee489a8-2573-480d-88ab-96e59d689de2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:36.598421 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee489a8-2573-480d-88ab-96e59d689de2 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:36.609818 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1451/2873] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:40:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:40:36.617542 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0636bf29-7337-4296-982e-fd7f1f36b74a admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:36.627900 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0636bf29-7337-4296-982e-fd7f1f36b74a admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:36.632696 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e88576a-45c4-4210-a355-727c84d51381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:36.633417 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e88576a-45c4-4210-a355-727c84d51381 admin admin] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:36.633867 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e88576a-45c4-4210-a355-727c84d51381 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:36.634399 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e88576a-45c4-4210-a355-727c84d51381 admin admin] PATH_INFO: `/v3/roles` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:36.635220 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e88576a-45c4-4210-a355-727c84d51381 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:36.636845 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e88576a-45c4-4210-a355-727c84d51381 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:36.647079 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1423/2874] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:40:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:40:36.652669 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee489a8-2573-480d-88ab-96e59d689de2 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:36.663682 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee489a8-2573-480d-88ab-96e59d689de2 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:36.669658 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86e0e34-0979-477f-be0c-62031651b90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:36.670526 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e0e34-0979-477f-be0c-62031651b90b admin admin] REQUEST_METHOD: `PUT` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:36.670806 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e0e34-0979-477f-be0c-62031651b90b admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:36.671079 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e0e34-0979-477f-be0c-62031651b90b admin admin] PATH_INFO: `/v3/projects/de887ac9630748ff80970b49454025fd/users/35dc91771b734cf8a4da79fa4971548e/roles/c467f10d3dba4f2dbfd42068f42f74b6` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:36.686996 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86e0e34-0979-477f-be0c-62031651b90b admin admin] RBAC: Authorizing `identity:create_grant(project_id=de887ac9630748ff80970b49454025fd, user_id=35dc91771b734cf8a4da79fa4971548e, role_id=c467f10d3dba4f2dbfd42068f42f74b6)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:36.689668 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86e0e34-0979-477f-be0c-62031651b90b admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:36.766669 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1452/2875] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:40:36 2022] PUT /identity/v3/projects/de887ac9630748ff80970b49454025fd/users/35dc91771b734cf8a4da79fa4971548e/roles/c467f10d3dba4f2dbfd42068f42f74b6 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:40:36.771830 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38441f4c-1c09-41ef-8aaf-285080b59e64 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:36.772124 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38441f4c-1c09-41ef-8aaf-285080b59e64 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:36.772431 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38441f4c-1c09-41ef-8aaf-285080b59e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:36.790800 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-38441f4c-1c09-41ef-8aaf-285080b59e64 None None] MFA Rules not processed for user `0c89ac265bd34c75a692d91d672fad59`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:40:36.792018 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-38441f4c-1c09-41ef-8aaf-285080b59e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:36.812782 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:40:36.852956 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1424/2876] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:40:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:40:36.863425 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86e0e34-0979-477f-be0c-62031651b90b admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:36.865152 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-c86e0e34-0979-477f-be0c-62031651b90b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:36.924849 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86e0e34-0979-477f-be0c-62031651b90b admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:36.928988 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da01bc-db1a-4983-b227-66e8ba66b20d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:36.929636 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da01bc-db1a-4983-b227-66e8ba66b20d service neutron] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:36.929995 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da01bc-db1a-4983-b227-66e8ba66b20d service neutron] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:36.930277 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da01bc-db1a-4983-b227-66e8ba66b20d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:36.939575 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79da01bc-db1a-4983-b227-66e8ba66b20d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:36.941116 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79da01bc-db1a-4983-b227-66e8ba66b20d service neutron] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:36.951618 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1453/2877] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:40:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:40:42.435034 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e5b33b-ef60-4e2d-afda-923b2cf91df2 None None] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:42.435561 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e5b33b-ef60-4e2d-afda-923b2cf91df2 None None] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:42.435801 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e5b33b-ef60-4e2d-afda-923b2cf91df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:42.467327 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.auth.core [None req-f9e5b33b-ef60-4e2d-afda-923b2cf91df2 None None] MFA Rules not processed for user `35dc91771b734cf8a4da79fa4971548e`. Rule list: `[]` (Enabled: `True`). {{(pid=80615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:40:42.468715 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-f9e5b33b-ef60-4e2d-afda-923b2cf91df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:42.538859 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1425/2878] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:40:42 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:40:42.543422 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da01bc-db1a-4983-b227-66e8ba66b20d service neutron] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:42.553408 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da01bc-db1a-4983-b227-66e8ba66b20d service neutron] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:42.558323 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1cba36-6070-4859-9e97-4bceac936d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:42.559007 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1cba36-6070-4859-9e97-4bceac936d27 admin admin] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:42.559299 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1cba36-6070-4859-9e97-4bceac936d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:42.559592 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1cba36-6070-4859-9e97-4bceac936d27 admin admin] PATH_INFO: `/v3/projects` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:42.560333 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1cba36-6070-4859-9e97-4bceac936d27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:42.562386 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1cba36-6070-4859-9e97-4bceac936d27 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:42.637563 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1454/2879] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jun 21 18:40:42 2022] POST /identity/v3/projects => generated 377 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:40:42.641790 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e5b33b-ef60-4e2d-afda-923b2cf91df2 None None] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:42.654101 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e5b33b-ef60-4e2d-afda-923b2cf91df2 None None] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:42.715476 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9da798-fabd-46e1-8300-8d47cb6ba8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:42.716169 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9da798-fabd-46e1-8300-8d47cb6ba8c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:42.716486 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9da798-fabd-46e1-8300-8d47cb6ba8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:42.716766 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9da798-fabd-46e1-8300-8d47cb6ba8c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:42.717472 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9da798-fabd-46e1-8300-8d47cb6ba8c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:42.719384 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9da798-fabd-46e1-8300-8d47cb6ba8c4 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:42.812858 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1426/2880] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jun 21 18:40:42 2022] POST /identity/v3/users => generated 295 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:40:42.816948 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1cba36-6070-4859-9e97-4bceac936d27 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:42.828478 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1cba36-6070-4859-9e97-4bceac936d27 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:42.833133 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a5871c-1985-41cb-aecf-21eae6067102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:42.833806 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a5871c-1985-41cb-aecf-21eae6067102 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:42.834093 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a5871c-1985-41cb-aecf-21eae6067102 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:42.834378 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a5871c-1985-41cb-aecf-21eae6067102 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:42.835201 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a5871c-1985-41cb-aecf-21eae6067102 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:42.837520 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a5871c-1985-41cb-aecf-21eae6067102 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:42.848536 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1455/2881] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:40:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:40:42.853010 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9da798-fabd-46e1-8300-8d47cb6ba8c4 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:42.863463 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9da798-fabd-46e1-8300-8d47cb6ba8c4 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:42.868714 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd76f07-326b-4101-868f-271c8cd89d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:42.869630 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd76f07-326b-4101-868f-271c8cd89d83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:42.869910 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd76f07-326b-4101-868f-271c8cd89d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:42.870186 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd76f07-326b-4101-868f-271c8cd89d83 admin admin] PATH_INFO: `/v3/projects/2f4ae9c9346b457dae8438446495d5b3/users/032ce7014cbf45879b3fd2d579f5c918/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:42.886712 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd76f07-326b-4101-868f-271c8cd89d83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f4ae9c9346b457dae8438446495d5b3, user_id=032ce7014cbf45879b3fd2d579f5c918, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:42.889411 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd76f07-326b-4101-868f-271c8cd89d83 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:42.962999 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1427/2882] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:40:42 2022] PUT /identity/v3/projects/2f4ae9c9346b457dae8438446495d5b3/users/032ce7014cbf45879b3fd2d579f5c918/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:40:42.967245 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a5871c-1985-41cb-aecf-21eae6067102 admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:42.977637 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a5871c-1985-41cb-aecf-21eae6067102 admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:43.027275 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:40:43.039003 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685cff64-1d61-4419-ac44-a194ebf92e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:43.039766 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685cff64-1d61-4419-ac44-a194ebf92e92 admin admin] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:43.040059 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685cff64-1d61-4419-ac44-a194ebf92e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:43.040335 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685cff64-1d61-4419-ac44-a194ebf92e92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:43.041153 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685cff64-1d61-4419-ac44-a194ebf92e92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:43.042753 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685cff64-1d61-4419-ac44-a194ebf92e92 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:43.052764 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1456/2883] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jun 21 18:40:42 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:40:43.057759 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd76f07-326b-4101-868f-271c8cd89d83 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:43.068373 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd76f07-326b-4101-868f-271c8cd89d83 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:43.073770 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65ec0bb-01a1-45c9-9f60-cdceeaf1defd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:43.074656 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ec0bb-01a1-45c9-9f60-cdceeaf1defd admin admin] REQUEST_METHOD: `PUT` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:43.074943 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ec0bb-01a1-45c9-9f60-cdceeaf1defd admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:43.075220 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ec0bb-01a1-45c9-9f60-cdceeaf1defd admin admin] PATH_INFO: `/v3/projects/2f4ae9c9346b457dae8438446495d5b3/users/032ce7014cbf45879b3fd2d579f5c918/roles/745ed9065a8143e384ebbd694b29635c` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:43.077223 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65ec0bb-01a1-45c9-9f60-cdceeaf1defd admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f4ae9c9346b457dae8438446495d5b3, user_id=032ce7014cbf45879b3fd2d579f5c918, role_id=745ed9065a8143e384ebbd694b29635c)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:43.079928 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65ec0bb-01a1-45c9-9f60-cdceeaf1defd admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:43.090463 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1428/2884] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jun 21 18:40:43 2022] PUT /identity/v3/projects/2f4ae9c9346b457dae8438446495d5b3/users/032ce7014cbf45879b3fd2d579f5c918/roles/745ed9065a8143e384ebbd694b29635c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:40:48.991858 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e227c-e2ce-4621-94fa-002bf2049d67 None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:48.992940 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e227c-e2ce-4621-94fa-002bf2049d67 None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:48.993040 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e227c-e2ce-4621-94fa-002bf2049d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:49.025121 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-8b6e227c-e2ce-4621-94fa-002bf2049d67 None None] MFA Rules not processed for user `032ce7014cbf45879b3fd2d579f5c918`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:40:49.026338 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-8b6e227c-e2ce-4621-94fa-002bf2049d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:49.096267 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1457/2885] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:40:48 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:40:49.113831 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65ec0bb-01a1-45c9-9f60-cdceeaf1defd admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:40:49.115560 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-e65ec0bb-01a1-45c9-9f60-cdceeaf1defd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:40:49.129918 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:40:49.221872 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65ec0bb-01a1-45c9-9f60-cdceeaf1defd admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:40:49.228627 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e76b5bd-9976-4635-9bfe-593035486224 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904255dd3c1f406eb10f6fc65f9cd098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:40:49.229299 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e76b5bd-9976-4635-9bfe-593035486224 service nova] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:40:49.229589 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e76b5bd-9976-4635-9bfe-593035486224 service nova] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:40:49.229880 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e76b5bd-9976-4635-9bfe-593035486224 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:40:49.286140 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e76b5bd-9976-4635-9bfe-593035486224 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:40:49.287737 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e76b5bd-9976-4635-9bfe-593035486224 service nova] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:40:49.297641 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1429/2886] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:40:49 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:40:59.093350 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=41 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:40:59.411616 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=64 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:41:07.593881 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea890e2-9d26-4a2b-9291-f7575fa431fb None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:07.594375 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea890e2-9d26-4a2b-9291-f7575fa431fb None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:07.594496 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea890e2-9d26-4a2b-9291-f7575fa431fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:07.612896 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-1ea890e2-9d26-4a2b-9291-f7575fa431fb None None] MFA Rules not processed for user `35dc91771b734cf8a4da79fa4971548e`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:41:07.614101 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-1ea890e2-9d26-4a2b-9291-f7575fa431fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:07.621412 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1458/2887] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:41:07 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:41:07.945776 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e76b5bd-9976-4635-9bfe-593035486224 service nova] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:08.001383 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e76b5bd-9976-4635-9bfe-593035486224 service nova] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:08.005090 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366e12c-e76a-40dc-85c7-f4349e7ec130 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:08.005735 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366e12c-e76a-40dc-85c7-f4349e7ec130 service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:08.006012 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366e12c-e76a-40dc-85c7-f4349e7ec130 service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:08.006285 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366e12c-e76a-40dc-85c7-f4349e7ec130 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:08.008036 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-e366e12c-e76a-40dc-85c7-f4349e7ec130 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:08.019696 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e366e12c-e76a-40dc-85c7-f4349e7ec130 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:08.021152 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e366e12c-e76a-40dc-85c7-f4349e7ec130 service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:08.030500 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1430/2888] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:41:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:41:17.607616 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=3 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:41:18.025496 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=26 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:41:25.280820 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea890e2-9d26-4a2b-9291-f7575fa431fb None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:41:25.282317 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-1ea890e2-9d26-4a2b-9291-f7575fa431fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:41:25.294715 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea890e2-9d26-4a2b-9291-f7575fa431fb None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:41:25.298875 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2286d866-c96f-4959-9526-378f0f15f8cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:41:25.299515 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2286d866-c96f-4959-9526-378f0f15f8cb service neutron] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:41:25.299799 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2286d866-c96f-4959-9526-378f0f15f8cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:41:25.300074 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2286d866-c96f-4959-9526-378f0f15f8cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:41:25.311493 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2286d866-c96f-4959-9526-378f0f15f8cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:41:25.313088 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2286d866-c96f-4959-9526-378f0f15f8cb service neutron] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:41:25.322700 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1459/2889] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:41:25 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:41:35.318412 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=6 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:43:23.152698 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366e12c-e76a-40dc-85c7-f4349e7ec130 service neutron] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:43:23.162158 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366e12c-e76a-40dc-85c7-f4349e7ec130 service neutron] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:43:23.165884 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738b845f-3bc3-4ab7-9c18-71e50a2beb5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:43:23.166481 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b845f-3bc3-4ab7-9c18-71e50a2beb5a service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:43:23.166766 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b845f-3bc3-4ab7-9c18-71e50a2beb5a service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:43:23.167041 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b845f-3bc3-4ab7-9c18-71e50a2beb5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:43:23.168830 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-738b845f-3bc3-4ab7-9c18-71e50a2beb5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:43:23.183010 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738b845f-3bc3-4ab7-9c18-71e50a2beb5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:43:23.184470 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738b845f-3bc3-4ab7-9c18-71e50a2beb5a service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:43:23.195586 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1431/2890] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:43:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:43:24.368932 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2286d866-c96f-4959-9526-378f0f15f8cb service neutron] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:43:24.378332 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2286d866-c96f-4959-9526-378f0f15f8cb service neutron] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:43:24.382004 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7642b0e2-f267-49c3-8947-95f62ddca4c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:43:24.385927 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7642b0e2-f267-49c3-8947-95f62ddca4c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:43:24.385927 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7642b0e2-f267-49c3-8947-95f62ddca4c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:43:24.385927 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7642b0e2-f267-49c3-8947-95f62ddca4c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:43:24.385927 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-7642b0e2-f267-49c3-8947-95f62ddca4c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:43:24.398925 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7642b0e2-f267-49c3-8947-95f62ddca4c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:43:24.400339 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7642b0e2-f267-49c3-8947-95f62ddca4c1 service neutron] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:43:24.411610 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1460/2891] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:43:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:43:25.506796 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738b845f-3bc3-4ab7-9c18-71e50a2beb5a service neutron] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:43:25.522734 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738b845f-3bc3-4ab7-9c18-71e50a2beb5a service neutron] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:43:25.528663 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb8e872-63a8-4db4-b3b9-ed35e3352070 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904255dd3c1f406eb10f6fc65f9cd098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:43:25.529379 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb8e872-63a8-4db4-b3b9-ed35e3352070 service nova] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:43:25.529986 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb8e872-63a8-4db4-b3b9-ed35e3352070 service nova] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:43:25.530274 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb8e872-63a8-4db4-b3b9-ed35e3352070 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:43:25.532040 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-3bb8e872-63a8-4db4-b3b9-ed35e3352070 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:43:25.549306 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb8e872-63a8-4db4-b3b9-ed35e3352070 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:43:25.550935 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb8e872-63a8-4db4-b3b9-ed35e3352070 service nova] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:43:25.563782 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1432/2892] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:43:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:43:27.842968 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b8e20d-b354-460d-a06b-265cb1461bba None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:43:27.843503 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b8e20d-b354-460d-a06b-265cb1461bba None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:43:27.843632 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b8e20d-b354-460d-a06b-265cb1461bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:43:27.861305 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-d2b8e20d-b354-460d-a06b-265cb1461bba None None] MFA Rules not processed for user `35dc91771b734cf8a4da79fa4971548e`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:43:27.862527 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-d2b8e20d-b354-460d-a06b-265cb1461bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:43:27.869572 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1461/2893] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:43:27 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:43:28.179424 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb8e872-63a8-4db4-b3b9-ed35e3352070 service nova] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:43:28.188743 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb8e872-63a8-4db4-b3b9-ed35e3352070 service nova] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:43:28.192987 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7186b83a-8151-422d-9c0d-299f44ccfd9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:43:28.193635 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7186b83a-8151-422d-9c0d-299f44ccfd9a service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:43:28.193923 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7186b83a-8151-422d-9c0d-299f44ccfd9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:43:28.194202 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7186b83a-8151-422d-9c0d-299f44ccfd9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:43:28.196541 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-7186b83a-8151-422d-9c0d-299f44ccfd9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:43:28.207276 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7186b83a-8151-422d-9c0d-299f44ccfd9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:43:28.209142 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7186b83a-8151-422d-9c0d-299f44ccfd9a service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:43:28.217415 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1433/2894] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:43:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:43:29.960530 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b8e20d-b354-460d-a06b-265cb1461bba None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:43:29.962271 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-d2b8e20d-b354-460d-a06b-265cb1461bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:43:30.034320 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b8e20d-b354-460d-a06b-265cb1461bba None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:43:30.039675 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fee5f9-294d-4851-a4e3-076105633dda service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938187967efe49be9c5389d1feef81ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:43:30.040343 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fee5f9-294d-4851-a4e3-076105633dda service placement] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:43:30.040661 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fee5f9-294d-4851-a4e3-076105633dda service placement] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:43:30.040948 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fee5f9-294d-4851-a4e3-076105633dda service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:43:30.042891 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-65fee5f9-294d-4851-a4e3-076105633dda service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:43:30.056075 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fee5f9-294d-4851-a4e3-076105633dda service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:43:30.057847 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fee5f9-294d-4851-a4e3-076105633dda service placement] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:43:30.069231 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1462/2895] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:43:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:43:35.935605 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7186b83a-8151-422d-9c0d-299f44ccfd9a service neutron] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:43:35.945405 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7186b83a-8151-422d-9c0d-299f44ccfd9a service neutron] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:43:35.950307 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcb77eb-8a5d-4363-9f08-09c805d1ee8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:43:35.951115 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcb77eb-8a5d-4363-9f08-09c805d1ee8d service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:43:35.951406 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcb77eb-8a5d-4363-9f08-09c805d1ee8d service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:43:35.951688 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcb77eb-8a5d-4363-9f08-09c805d1ee8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:43:35.954244 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-3fcb77eb-8a5d-4363-9f08-09c805d1ee8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:43:35.965674 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcb77eb-8a5d-4363-9f08-09c805d1ee8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:43:35.967282 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcb77eb-8a5d-4363-9f08-09c805d1ee8d service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:43:35.977571 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1434/2896] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:43:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:43:40.063624 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=41 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:43:45.973266 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=24 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:43:47.775922 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fee5f9-294d-4851-a4e3-076105633dda service placement] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:43:47.777453 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-65fee5f9-294d-4851-a4e3-076105633dda service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:43:47.793770 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fee5f9-294d-4851-a4e3-076105633dda service placement] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:43:47.799382 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a59ddf3-580c-4fb5-9f85-98053afa1c80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904255dd3c1f406eb10f6fc65f9cd098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:43:47.800792 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a59ddf3-580c-4fb5-9f85-98053afa1c80 service nova] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:43:47.801114 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a59ddf3-580c-4fb5-9f85-98053afa1c80 service nova] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:43:47.802647 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a59ddf3-580c-4fb5-9f85-98053afa1c80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:43:47.803233 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-7a59ddf3-580c-4fb5-9f85-98053afa1c80 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:43:47.815650 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a59ddf3-580c-4fb5-9f85-98053afa1c80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:43:47.817255 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a59ddf3-580c-4fb5-9f85-98053afa1c80 service nova] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:43:47.828064 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1463/2897] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:43:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:43:57.823397 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=6 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:45:17.128979 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcb77eb-8a5d-4363-9f08-09c805d1ee8d service neutron] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:45:17.130480 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-3fcb77eb-8a5d-4363-9f08-09c805d1ee8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:45:17.146481 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcb77eb-8a5d-4363-9f08-09c805d1ee8d service neutron] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:45:17.151370 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fc452a-9c7e-4540-8db3-0b0b596582df service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938187967efe49be9c5389d1feef81ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:45:17.152013 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc452a-9c7e-4540-8db3-0b0b596582df service placement] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:45:17.152436 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc452a-9c7e-4540-8db3-0b0b596582df service placement] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:45:17.152720 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc452a-9c7e-4540-8db3-0b0b596582df service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:45:17.154454 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-79fc452a-9c7e-4540-8db3-0b0b596582df service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:45:17.167364 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fc452a-9c7e-4540-8db3-0b0b596582df service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:45:17.168987 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fc452a-9c7e-4540-8db3-0b0b596582df service placement] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:45:17.179690 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1435/2898] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:45:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:45:26.166004 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11056adb-93ca-46a5-8164-658e8edbbcbf None None] REQUEST_METHOD: `POST` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:45:26.166549 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11056adb-93ca-46a5-8164-658e8edbbcbf None None] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:45:26.166549 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11056adb-93ca-46a5-8164-658e8edbbcbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:45:26.185430 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.auth.core [None req-11056adb-93ca-46a5-8164-658e8edbbcbf None None] MFA Rules not processed for user `35dc91771b734cf8a4da79fa4971548e`. Rule list: `[]` (Enabled: `True`). {{(pid=80614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:45:26.187444 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-11056adb-93ca-46a5-8164-658e8edbbcbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:45:26.194459 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1464/2899] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jun 21 18:45:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:45:27.175391 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=6 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:45:36.179908 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=3 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:47:09.079806 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fc452a-9c7e-4540-8db3-0b0b596582df service placement] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:47:09.089469 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fc452a-9c7e-4540-8db3-0b0b596582df service placement] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:47:09.093297 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75d0a33-5c68-4ed2-a444-24e98459dcad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:47:09.093951 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d0a33-5c68-4ed2-a444-24e98459dcad service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:09.094233 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d0a33-5c68-4ed2-a444-24e98459dcad service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:09.094505 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d0a33-5c68-4ed2-a444-24e98459dcad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:09.103695 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75d0a33-5c68-4ed2-a444-24e98459dcad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:47:09.105130 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75d0a33-5c68-4ed2-a444-24e98459dcad service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:47:09.113667 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1436/2900] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:47:09 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:47:10.179000 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11056adb-93ca-46a5-8164-658e8edbbcbf None None] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:47:10.198899 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11056adb-93ca-46a5-8164-658e8edbbcbf None None] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:47:10.204799 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5185237-dc77-49cf-8893-9ddf3f8899f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904255dd3c1f406eb10f6fc65f9cd098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:47:10.205533 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5185237-dc77-49cf-8893-9ddf3f8899f7 service nova] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:10.205820 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5185237-dc77-49cf-8893-9ddf3f8899f7 service nova] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:10.206104 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5185237-dc77-49cf-8893-9ddf3f8899f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:10.208142 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-c5185237-dc77-49cf-8893-9ddf3f8899f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:47:10.222028 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5185237-dc77-49cf-8893-9ddf3f8899f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:47:10.223730 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5185237-dc77-49cf-8893-9ddf3f8899f7 service nova] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:47:10.237439 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1465/2901] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:47:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:47:19.109779 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=6 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:47:20.232843 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=6 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:47:23.564568 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75d0a33-5c68-4ed2-a444-24e98459dcad service neutron] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:47:23.573969 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75d0a33-5c68-4ed2-a444-24e98459dcad service neutron] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:47:23.577689 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235753c0-fd40-4581-9e13-40d4dcb5fd8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c7f07e58ba4db9a7d402d8f3da1e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:47:23.578333 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235753c0-fd40-4581-9e13-40d4dcb5fd8a service neutron] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:23.578620 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235753c0-fd40-4581-9e13-40d4dcb5fd8a service neutron] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:23.578900 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235753c0-fd40-4581-9e13-40d4dcb5fd8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:23.639735 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235753c0-fd40-4581-9e13-40d4dcb5fd8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:47:23.641468 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235753c0-fd40-4581-9e13-40d4dcb5fd8a service neutron] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:47:23.651786 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1437/2902] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:47:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:47:26.872364 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5185237-dc77-49cf-8893-9ddf3f8899f7 service nova] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:47:26.884074 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5185237-dc77-49cf-8893-9ddf3f8899f7 service nova] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:47:26.892375 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d25ff7-150f-47e7-ac29-da72ad390945 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938187967efe49be9c5389d1feef81ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:47:26.893066 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d25ff7-150f-47e7-ac29-da72ad390945 service placement] REQUEST_METHOD: `GET` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:26.893351 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d25ff7-150f-47e7-ac29-da72ad390945 service placement] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:26.893629 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d25ff7-150f-47e7-ac29-da72ad390945 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:26.895392 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-23d25ff7-150f-47e7-ac29-da72ad390945 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:47:26.907623 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d25ff7-150f-47e7-ac29-da72ad390945 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:47:26.909323 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d25ff7-150f-47e7-ac29-da72ad390945 service placement] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:47:26.920927 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1466/2903] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:47:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:47:30.695040 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235753c0-fd40-4581-9e13-40d4dcb5fd8a service neutron] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:47:30.706457 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235753c0-fd40-4581-9e13-40d4dcb5fd8a service neutron] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:47:30.711243 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6463995a-0103-4860-ad4d-30c89f602d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:47:30.712130 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6463995a-0103-4860-ad4d-30c89f602d87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:30.712477 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6463995a-0103-4860-ad4d-30c89f602d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:30.717020 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6463995a-0103-4860-ad4d-30c89f602d87 admin admin] PATH_INFO: `/v3/users/35dc91771b734cf8a4da79fa4971548e` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:30.717020 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6463995a-0103-4860-ad4d-30c89f602d87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35dc91771b734cf8a4da79fa4971548e)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:47:30.717020 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6463995a-0103-4860-ad4d-30c89f602d87 admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:47:30.882949 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-6463995a-0103-4860-ad4d-30c89f602d87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35dc91771b734cf8a4da79fa4971548e'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:30.895680 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-6463995a-0103-4860-ad4d-30c89f602d87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35dc91771b734cf8a4da79fa4971548e'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:30.896419 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-6463995a-0103-4860-ad4d-30c89f602d87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35dc91771b734cf8a4da79fa4971548e'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:30.903913 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-6463995a-0103-4860-ad4d-30c89f602d87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35dc91771b734cf8a4da79fa4971548e'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:30.904581 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-6463995a-0103-4860-ad4d-30c89f602d87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35dc91771b734cf8a4da79fa4971548e'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:30.992071 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1438/2904] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:47:30 2022] DELETE /identity/v3/users/35dc91771b734cf8a4da79fa4971548e => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:47:31.210951 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d25ff7-150f-47e7-ac29-da72ad390945 service placement] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:47:31.222923 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d25ff7-150f-47e7-ac29-da72ad390945 service placement] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:47:31.282552 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f944e7e-c9c4-493a-959a-00f50c4edcaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:47:31.283447 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f944e7e-c9c4-493a-959a-00f50c4edcaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:31.283703 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f944e7e-c9c4-493a-959a-00f50c4edcaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:31.283972 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f944e7e-c9c4-493a-959a-00f50c4edcaf admin admin] PATH_INFO: `/v3/projects/de887ac9630748ff80970b49454025fd` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:31.285209 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f944e7e-c9c4-493a-959a-00f50c4edcaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=de887ac9630748ff80970b49454025fd)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:47:31.287113 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f944e7e-c9c4-493a-959a-00f50c4edcaf admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:47:31.404303 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-3f944e7e-c9c4-493a-959a-00f50c4edcaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de887ac9630748ff80970b49454025fd', 'request_id': 'req-3f944e7e-c9c4-493a-959a-00f50c4edcaf'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:31.410586 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-3f944e7e-c9c4-493a-959a-00f50c4edcaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de887ac9630748ff80970b49454025fd', 'request_id': 'req-3f944e7e-c9c4-493a-959a-00f50c4edcaf'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:31.416328 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-3f944e7e-c9c4-493a-959a-00f50c4edcaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de887ac9630748ff80970b49454025fd', 'request_id': 'req-3f944e7e-c9c4-493a-959a-00f50c4edcaf'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:31.481150 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-3f944e7e-c9c4-493a-959a-00f50c4edcaf admin admin] The token cache is being invalidate because project de887ac9630748ff80970b49454025fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:47:31.481461 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-3f944e7e-c9c4-493a-959a-00f50c4edcaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:31.482097 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-3f944e7e-c9c4-493a-959a-00f50c4edcaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:31.483797 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1467/2905] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:47:31 2022] DELETE /identity/v3/projects/de887ac9630748ff80970b49454025fd => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:47:31.487986 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6463995a-0103-4860-ad4d-30c89f602d87 admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:47:31.489408 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-6463995a-0103-4860-ad4d-30c89f602d87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:47:31.503848 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6463995a-0103-4860-ad4d-30c89f602d87 admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:47:31.564934 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678a8889-8574-4225-bb9d-3f8f2b42809d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:47:31.565876 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678a8889-8574-4225-bb9d-3f8f2b42809d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:31.566153 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678a8889-8574-4225-bb9d-3f8f2b42809d admin admin] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:31.566428 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678a8889-8574-4225-bb9d-3f8f2b42809d admin admin] PATH_INFO: `/v3/users/032ce7014cbf45879b3fd2d579f5c918` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:31.567620 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678a8889-8574-4225-bb9d-3f8f2b42809d admin admin] RBAC: Authorizing `identity:delete_user(user_id=032ce7014cbf45879b3fd2d579f5c918)` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:47:31.569556 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678a8889-8574-4225-bb9d-3f8f2b42809d admin admin] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:47:31.575868 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:47:31.737258 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-678a8889-8574-4225-bb9d-3f8f2b42809d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '032ce7014cbf45879b3fd2d579f5c918'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:31.749567 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-678a8889-8574-4225-bb9d-3f8f2b42809d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '032ce7014cbf45879b3fd2d579f5c918'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:31.750249 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-678a8889-8574-4225-bb9d-3f8f2b42809d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '032ce7014cbf45879b3fd2d579f5c918'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:31.757007 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-678a8889-8574-4225-bb9d-3f8f2b42809d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '032ce7014cbf45879b3fd2d579f5c918'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:31.757579 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.notifications [None req-678a8889-8574-4225-bb9d-3f8f2b42809d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '032ce7014cbf45879b3fd2d579f5c918'} {{(pid=80615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:31.825432 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1439/2906] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jun 21 18:47:31 2022] DELETE /identity/v3/users/032ce7014cbf45879b3fd2d579f5c918 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:47:32.104337 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f944e7e-c9c4-493a-959a-00f50c4edcaf admin admin] Authenticating user token {{(pid=80614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:47:32.104337 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.fernet_utils [None req-3f944e7e-c9c4-493a-959a-00f50c4edcaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:47:32.119064 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f944e7e-c9c4-493a-959a-00f50c4edcaf admin admin] Validating token access rules against request {{(pid=80614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:47:32.176869 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddba0ffa-27d4-41c0-8377-9cb90b9ddff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c89ac265bd34c75a692d91d672fad59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecad7a8a62644cabfc30fc28ce80d90', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:47:32.177755 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba0ffa-27d4-41c0-8377-9cb90b9ddff3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:47:32.178036 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba0ffa-27d4-41c0-8377-9cb90b9ddff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:47:32.178312 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba0ffa-27d4-41c0-8377-9cb90b9ddff3 admin admin] PATH_INFO: `/v3/projects/2f4ae9c9346b457dae8438446495d5b3` {{(pid=80614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:47:32.179478 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddba0ffa-27d4-41c0-8377-9cb90b9ddff3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f4ae9c9346b457dae8438446495d5b3)` {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:47:32.181786 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddba0ffa-27d4-41c0-8377-9cb90b9ddff3 admin admin] RBAC: Authorization granted {{(pid=80614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:47:32.202584 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:47:32.300142 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-ddba0ffa-27d4-41c0-8377-9cb90b9ddff3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f4ae9c9346b457dae8438446495d5b3', 'request_id': 'req-ddba0ffa-27d4-41c0-8377-9cb90b9ddff3'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:32.306535 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-ddba0ffa-27d4-41c0-8377-9cb90b9ddff3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f4ae9c9346b457dae8438446495d5b3', 'request_id': 'req-ddba0ffa-27d4-41c0-8377-9cb90b9ddff3'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:32.311949 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-ddba0ffa-27d4-41c0-8377-9cb90b9ddff3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f4ae9c9346b457dae8438446495d5b3', 'request_id': 'req-ddba0ffa-27d4-41c0-8377-9cb90b9ddff3'} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:32.406774 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-ddba0ffa-27d4-41c0-8377-9cb90b9ddff3 admin admin] The token cache is being invalidate because project 2f4ae9c9346b457dae8438446495d5b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:47:32.410622 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-ddba0ffa-27d4-41c0-8377-9cb90b9ddff3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:32.410622 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG keystone.notifications [None req-ddba0ffa-27d4-41c0-8377-9cb90b9ddff3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:47:32.410622 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: [pid: 80614|app: 0|req: 1468/2907] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jun 21 18:47:32 2022] DELETE /identity/v3/projects/2f4ae9c9346b457dae8438446495d5b3 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:47:41.821750 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:47:42.458463 dv-cinder-831247-19-iscsi devstack@keystone.service[80614]: DEBUG dbcounter [-] [80614] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:20.480206 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678a8889-8574-4225-bb9d-3f8f2b42809d admin admin] Authenticating user token {{(pid=80615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:20.481694 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-678a8889-8574-4225-bb9d-3f8f2b42809d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:20.619623 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678a8889-8574-4225-bb9d-3f8f2b42809d admin admin] Validating token access rules against request {{(pid=80615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:20.676786 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d84f1bc-23be-4a34-becd-954345975dae service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938187967efe49be9c5389d1feef81ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a4699d1ec70456da5f8e0aa4b1439b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:20.677447 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d84f1bc-23be-4a34-becd-954345975dae service placement] REQUEST_METHOD: `GET` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:20.677735 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d84f1bc-23be-4a34-becd-954345975dae service placement] SCRIPT_NAME: `/identity` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:20.678048 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d84f1bc-23be-4a34-becd-954345975dae service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:20.680549 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.fernet_utils [None req-9d84f1bc-23be-4a34-becd-954345975dae service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:20.691943 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d84f1bc-23be-4a34-becd-954345975dae service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:20.693554 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d84f1bc-23be-4a34-becd-954345975dae service placement] RBAC: Authorization granted {{(pid=80615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:20.703793 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: [pid: 80615|app: 0|req: 1440/2908] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jun 21 18:50:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:50:30.699383 dv-cinder-831247-19-iscsi devstack@keystone.service[80615]: DEBUG dbcounter [-] [80615] Writing DB stats keystone:SELECT=89 {{(pid=80615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}